[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790257629041645-13911-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 13:47:09 dist-test-slave-sjl2 krb5kdc[13918](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 13:47:09 dist-test-slave-sjl2 krb5kdc[13918](info): set up 2 sockets
Sep 24 13:47:09 dist-test-slave-sjl2 krb5kdc[13918](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260924 13:47:09.998227 13911 mini_kdc.cc:121] Time spent starting KDC: real 0.939s user 0.004s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790257629041645-13911-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790257629041645-13911-0/krb5kdc/test-user.keytab.
Sep 24 13:47:10 dist-test-slave-sjl2 krb5kdc[13918](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257630, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T13:47:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T13:47:10Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.149.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.149.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.149.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.254.keytab.
Entry for principal kudu/127.13.149.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.254.keytab.
I20260924 13:47:10.120800 13911 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJN9WRF/build/release/bin/kudu
/tmp/dist-test-taskJN9WRF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790257629041645-13911-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790257629041645-13911-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790257629041645-13911-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790257629041645-13911-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.149.254:37959
--webserver_interface=127.13.149.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.254.keytab
--principal=kudu/127.13.149.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.149.212:40499
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.149.254:37959
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790257629041645-13911-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790257629041645-13911-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790257629041645-13911-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 13:47:10.200614 13931 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 13:47:10.200838 13931 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 13:47:10.200860 13931 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 13:47:10.202026 13931 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 13:47:10.202066 13931 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 13:47:10.202080 13931 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 13:47:10.202091 13931 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 13:47:10.202111 13931 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 13:47:10.203598 13931 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.149.212:40499
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790257629041645-13911-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790257629041645-13911-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.149.254:37959
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.254.keytab
--principal=kudu/127.13.149.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.149.254:37959
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790257629041645-13911-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.149.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13931
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790257629041645-13911-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 12:29:37 UTC on ccycloud.quasar-iqzbxn.root.comops.site
build id 4670106
I20260924 13:47:10.203827 13931 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 13:47:10.204026 13931 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 13:47:10.209089 13939 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 13:47:10.209101 13937 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 13:47:10.209101 13936 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 13:47:10.209143 13931 server_base.cc:1034] running on GCE node
I20260924 13:47:10.209503 13931 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 13:47:10.209920 13931 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 13:47:10.211068 13931 hybrid_clock.cc:648] HybridClock initialized: now 1790257630211048 us; error 33 us; skew 500 ppm
Sep 24 13:47:10 dist-test-slave-sjl2 krb5kdc[13918](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257630, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.149.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 13:47:10.213546 13931 init.cc:377] Logged in from keytab as kudu/127.13.149.254@KRBTEST.COM (short username kudu)
I20260924 13:47:10.213991 13931 webserver.cc:460] Webserver started at http://127.13.149.254:34391/ using document root <none> and password file <none>
I20260924 13:47:10.214205 13931 fs_manager.cc:359] Metadata directory not provided
I20260924 13:47:10.214249 13931 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 13:47:10.214389 13931 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 13:47:10.215456 13931 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790257629041645-13911-0/minicluster-data/master-0/data/instance:
uuid: "39ce3fba63c8440da0d570c7edfcaaca"
format_stamp: "Formatted at 2026-09-24 13:47:10 on dist-test-slave-sjl2"
server_key: "18ecd3fe2ec93fda2b07310a9baddd2f"
server_key_iv: "2582d5e5ed695357c2276c6bfe26455c"
server_key_version: "encryptionkey@0"
I20260924 13:47:10.215790 13931 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790257629041645-13911-0/minicluster-data/master-0/wal/instance:
uuid: "39ce3fba63c8440da0d570c7edfcaaca"
format_stamp: "Formatted at 2026-09-24 13:47:10 on dist-test-slave-sjl2"
server_key: "18ecd3fe2ec93fda2b07310a9baddd2f"
server_key_iv: "2582d5e5ed695357c2276c6bfe26455c"
server_key_version: "encryptionkey@0"
I20260924 13:47:10.217082 13931 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260924 13:47:10.217763 13946 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 13:47:10.217911 13931 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 13:47:10.217988 13931 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790257629041645-13911-0/minicluster-data/master-0/data,/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790257629041645-13911-0/minicluster-data/master-0/wal
uuid: "39ce3fba63c8440da0d570c7edfcaaca"
format_stamp: "Formatted at 2026-09-24 13:47:10 on dist-test-slave-sjl2"
server_key: "18ecd3fe2ec93fda2b07310a9baddd2f"
server_key_iv: "2582d5e5ed695357c2276c6bfe26455c"
server_key_version: "encryptionkey@0"
I20260924 13:47:10.218057 13931 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790257629041645-13911-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790257629041645-13911-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790257629041645-13911-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 13:47:10.231326 13931 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 13:47:10.233289 13931 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 13:47:10.233415 13931 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 13:47:10.237188 13931 rpc_server.cc:307] RPC server started. Bound to: 127.13.149.254:37959
I20260924 13:47:10.237234 13999 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.149.254:37959 every 8 connection(s)
I20260924 13:47:10.237569 13931 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790257629041645-13911-0/minicluster-data/master-0/data/info.pb
I20260924 13:47:10.238435 14000 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 13:47:10.240793 14000 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 39ce3fba63c8440da0d570c7edfcaaca: Bootstrap starting.
I20260924 13:47:10.241397 14000 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 39ce3fba63c8440da0d570c7edfcaaca: Neither blocks nor log segments found. Creating new log.
I20260924 13:47:10.241681 14000 log.cc:824] T 00000000000000000000000000000000 P 39ce3fba63c8440da0d570c7edfcaaca: Log is configured to *not* fsync() on all Append() calls
I20260924 13:47:10.242401 14000 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 39ce3fba63c8440da0d570c7edfcaaca: No bootstrap required, opened a new log
I20260924 13:47:10.244074 14000 raft_consensus.cc:348] T 00000000000000000000000000000000 P 39ce3fba63c8440da0d570c7edfcaaca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "39ce3fba63c8440da0d570c7edfcaaca" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 37959 } }
I20260924 13:47:10.244225 14000 raft_consensus.cc:374] T 00000000000000000000000000000000 P 39ce3fba63c8440da0d570c7edfcaaca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 13:47:10.244249 14000 raft_consensus.cc:729] T 00000000000000000000000000000000 P 39ce3fba63c8440da0d570c7edfcaaca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 39ce3fba63c8440da0d570c7edfcaaca, State: Initialized, Role: FOLLOWER
I20260924 13:47:10.244382 14000 consensus_queue.cc:260] T 00000000000000000000000000000000 P 39ce3fba63c8440da0d570c7edfcaaca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "39ce3fba63c8440da0d570c7edfcaaca" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 37959 } }
I20260924 13:47:10.244452 14000 raft_consensus.cc:388] T 00000000000000000000000000000000 P 39ce3fba63c8440da0d570c7edfcaaca [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 13:47:10.244473 14000 raft_consensus.cc:482] T 00000000000000000000000000000000 P 39ce3fba63c8440da0d570c7edfcaaca [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 13:47:10.244501 14000 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 39ce3fba63c8440da0d570c7edfcaaca [term 0 FOLLOWER]: Advancing to term 1
I20260924 13:47:10.245064 14000 raft_consensus.cc:504] T 00000000000000000000000000000000 P 39ce3fba63c8440da0d570c7edfcaaca [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "39ce3fba63c8440da0d570c7edfcaaca" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 37959 } }
I20260924 13:47:10.245154 14000 leader_election.cc:302] T 00000000000000000000000000000000 P 39ce3fba63c8440da0d570c7edfcaaca [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 39ce3fba63c8440da0d570c7edfcaaca; no voters:
I20260924 13:47:10.245319 14000 leader_election.cc:288] T 00000000000000000000000000000000 P 39ce3fba63c8440da0d570c7edfcaaca [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 13:47:10.245380 14003 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 39ce3fba63c8440da0d570c7edfcaaca [term 1 FOLLOWER]: Leader election won for term 1
I20260924 13:47:10.245565 14003 raft_consensus.cc:686] T 00000000000000000000000000000000 P 39ce3fba63c8440da0d570c7edfcaaca [term 1 LEADER]: Becoming Leader. State: Replica: 39ce3fba63c8440da0d570c7edfcaaca, State: Running, Role: LEADER
I20260924 13:47:10.245689 14003 consensus_queue.cc:237] T 00000000000000000000000000000000 P 39ce3fba63c8440da0d570c7edfcaaca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "39ce3fba63c8440da0d570c7edfcaaca" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 37959 } }
I20260924 13:47:10.245834 13911 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJN9WRF/build/release/bin/kudu as pid 13931
I20260924 13:47:10.245930 13911 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790257629041645-13911-0/minicluster-data/master-0/wal/instance
I20260924 13:47:10.246065 14004 sys_catalog.cc:455] T 00000000000000000000000000000000 P 39ce3fba63c8440da0d570c7edfcaaca [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "39ce3fba63c8440da0d570c7edfcaaca" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "39ce3fba63c8440da0d570c7edfcaaca" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 37959 } } }
I20260924 13:47:10.246132 14004 sys_catalog.cc:458] T 00000000000000000000000000000000 P 39ce3fba63c8440da0d570c7edfcaaca [sys.catalog]: This master's current role is: LEADER
I20260924 13:47:10.246236 14005 sys_catalog.cc:455] T 00000000000000000000000000000000 P 39ce3fba63c8440da0d570c7edfcaaca [sys.catalog]: SysCatalogTable state changed. Reason: New leader 39ce3fba63c8440da0d570c7edfcaaca. Latest consensus state: current_term: 1 leader_uuid: "39ce3fba63c8440da0d570c7edfcaaca" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "39ce3fba63c8440da0d570c7edfcaaca" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 37959 } } }
I20260924 13:47:10.246312 14005 sys_catalog.cc:458] T 00000000000000000000000000000000 P 39ce3fba63c8440da0d570c7edfcaaca [sys.catalog]: This master's current role is: LEADER
I20260924 13:47:10.246639 13911 external_mini_cluster.cc:1428] Setting key 32c6f9d404e315f0012d1b20b187f705
I20260924 13:47:10.247164 14000 sys_catalog.cc:564] T 00000000000000000000000000000000 P 39ce3fba63c8440da0d570c7edfcaaca [sys.catalog]: configured and running, proceeding with master startup.
I20260924 13:47:10.247262 14007 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 13:47:10.247783 14007 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 13:47:10.249572 14007 catalog_manager.cc:1347] Generated new cluster ID: 3be32652ff204e43bb30efcec125fae4
I20260924 13:47:10.249627 14007 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 24 13:47:10 dist-test-slave-sjl2 krb5kdc[13918](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257630, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.149.254@KRBTEST.COM
I20260924 13:47:10.264528 14015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:10.247908 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58764 (local address 127.13.149.254:37959)
0924 13:47:10.248089 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0924 13:47:10.248099 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0924 13:47:10.248299 (+ 200us) server_negotiation.cc:407] Connection header received
0924 13:47:10.250466 (+ 2167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:47:10.250473 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:47:10.250528 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:47:10.250680 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:47:10.252519 (+ 1839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:10.253992 (+ 1473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:10.255444 (+ 1452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:10.256099 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:10.256217 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:47:10.260295 (+ 4078us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:47:10.260301 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:47:10.260301 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 13:47:10.260314 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:47:10.262312 (+ 1998us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:47:10.262780 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:47:10.262784 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:47:10.262784 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:47:10.262898 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:47:10.263141 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:47:10.263141 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:47:10.263142 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:47:10.263482 (+ 340us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:47:10.263626 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0924 13:47:10.263901 (+ 275us) server_negotiation.cc:299] Negotiation successful
0924 13:47:10.264043 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":118,"thread_start_us":70,"threads_started":1}
I20260924 13:47:10.290438 14007 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 13:47:10.290978 14007 catalog_manager.cc:1504] Loading token signing keys...
I20260924 13:47:10.300554 14007 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 39ce3fba63c8440da0d570c7edfcaaca: Generated new TSK 0
I20260924 13:47:10.300782 14007 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.13.149.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.149.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.149.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.193.keytab.
Entry for principal kudu/127.13.149.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.193.keytab.
I20260924 13:47:10.331703 13911 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJN9WRF/build/release/bin/kudu
/tmp/dist-test-taskJN9WRF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790257629041645-13911-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790257629041645-13911-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790257629041645-13911-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790257629041645-13911-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.149.193:0
--local_ip_for_outbound_sockets=127.13.149.193
--webserver_interface=127.13.149.193
--webserver_port=0
--tserver_master_addrs=127.13.149.254:37959
--keytab_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.193.keytab
--principal=kudu/127.13.149.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.149.212:40499
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790257629041645-13911-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790257629041645-13911-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790257629041645-13911-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 13:47:10.412084 14026 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 13:47:10.412281 14026 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 13:47:10.412303 14026 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 13:47:10.413476 14026 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 13:47:10.413515 14026 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 13:47:10.413546 14026 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.149.193
I20260924 13:47:10.414983 14026 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.149.212:40499
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790257629041645-13911-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790257629041645-13911-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.193.keytab
--principal=kudu/127.13.149.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.149.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790257629041645-13911-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.149.193
--webserver_port=0
--tserver_master_addrs=127.13.149.254:37959
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14026
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.149.193
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790257629041645-13911-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 12:29:37 UTC on ccycloud.quasar-iqzbxn.root.comops.site
build id 4670106
I20260924 13:47:10.415215 14026 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 13:47:10.415438 14026 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 13:47:10.420358 14031 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 13:47:10.420353 14034 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 13:47:10.420353 14032 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 13:47:10.420473 14026 server_base.cc:1034] running on GCE node
I20260924 13:47:10.420790 14026 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 13:47:10.421083 14026 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 13:47:10.422240 14026 hybrid_clock.cc:648] HybridClock initialized: now 1790257630422226 us; error 28 us; skew 500 ppm
Sep 24 13:47:10 dist-test-slave-sjl2 krb5kdc[13918](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257630, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.149.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 13:47:10.424685 14026 init.cc:377] Logged in from keytab as kudu/127.13.149.193@KRBTEST.COM (short username kudu)
I20260924 13:47:10.425155 14026 webserver.cc:460] Webserver started at http://127.13.149.193:36707/ using document root <none> and password file <none>
I20260924 13:47:10.425382 14026 fs_manager.cc:359] Metadata directory not provided
I20260924 13:47:10.425431 14026 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 13:47:10.425561 14026 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 13:47:10.426625 14026 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790257629041645-13911-0/minicluster-data/ts-0/data/instance:
uuid: "3f3e6a719eaa4fac95cedbe8986edc49"
format_stamp: "Formatted at 2026-09-24 13:47:10 on dist-test-slave-sjl2"
server_key: "aa0d4c02669645457b57b20ceb57e5fc"
server_key_iv: "ed50cf79d941ff5ddbdeeecbb0976c23"
server_key_version: "encryptionkey@0"
I20260924 13:47:10.426963 14026 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790257629041645-13911-0/minicluster-data/ts-0/wal/instance:
uuid: "3f3e6a719eaa4fac95cedbe8986edc49"
format_stamp: "Formatted at 2026-09-24 13:47:10 on dist-test-slave-sjl2"
server_key: "aa0d4c02669645457b57b20ceb57e5fc"
server_key_iv: "ed50cf79d941ff5ddbdeeecbb0976c23"
server_key_version: "encryptionkey@0"
I20260924 13:47:10.428161 14026 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260924 13:47:10.428864 14041 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 13:47:10.429049 14026 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 13:47:10.429128 14026 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790257629041645-13911-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790257629041645-13911-0/minicluster-data/ts-0/wal
uuid: "3f3e6a719eaa4fac95cedbe8986edc49"
format_stamp: "Formatted at 2026-09-24 13:47:10 on dist-test-slave-sjl2"
server_key: "aa0d4c02669645457b57b20ceb57e5fc"
server_key_iv: "ed50cf79d941ff5ddbdeeecbb0976c23"
server_key_version: "encryptionkey@0"
I20260924 13:47:10.429181 14026 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790257629041645-13911-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790257629041645-13911-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790257629041645-13911-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 13:47:10.441134 14026 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 13:47:10.442802 14026 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 13:47:10.442934 14026 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 13:47:10.443131 14026 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 13:47:10.443388 14026 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 13:47:10.443440 14026 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 13:47:10.443462 14026 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 13:47:10.443475 14026 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 13:47:10.449092 14026 rpc_server.cc:307] RPC server started. Bound to: 127.13.149.193:45581
I20260924 13:47:10.449162 14155 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.149.193:45581 every 8 connection(s)
I20260924 13:47:10.449466 14026 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790257629041645-13911-0/minicluster-data/ts-0/data/info.pb
Sep 24 13:47:10 dist-test-slave-sjl2 krb5kdc[13918](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257630, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.149.193@KRBTEST.COM for kudu/127.13.149.254@KRBTEST.COM
I20260924 13:47:10.456343 13911 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJN9WRF/build/release/bin/kudu as pid 14026
I20260924 13:47:10.456434 13911 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790257629041645-13911-0/minicluster-data/ts-0/wal/instance
I20260924 13:47:10.456585 13911 external_mini_cluster.cc:1428] Setting key 802766284cbc6f6f517d9826c17dcfd6
I20260924 13:47:10.458396 14015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:10.450204 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.149.193:40055 (local address 127.13.149.254:37959)
0924 13:47:10.450282 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0924 13:47:10.450283 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0924 13:47:10.450455 (+ 172us) server_negotiation.cc:407] Connection header received
0924 13:47:10.450936 (+ 481us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:47:10.450937 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:47:10.450953 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:47:10.451029 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:47:10.451976 (+ 947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:10.452482 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:10.453563 (+ 1081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:10.453767 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:10.453834 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:47:10.455794 (+ 1960us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:47:10.455795 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:47:10.455796 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 13:47:10.455802 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:47:10.457195 (+ 1393us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:47:10.457570 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:47:10.457571 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:47:10.457571 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:47:10.457634 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:47:10.457826 (+ 192us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:47:10.457827 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:47:10.457827 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:47:10.457950 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:47:10.458012 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0924 13:47:10.458225 (+ 213us) server_negotiation.cc:299] Negotiation successful
0924 13:47:10.458307 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260924 13:47:10.458562 14158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:10.450152 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:37959 (local address 127.13.149.193:40055)
0924 13:47:10.450315 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0924 13:47:10.450359 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0924 13:47:10.450844 (+ 485us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:47:10.451120 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:47:10.451123 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:47:10.451689 (+ 566us) client_negotiation.cc:189] Negotiated authn=SASL
0924 13:47:10.451892 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:47:10.451896 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:47:10.452562 (+ 666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:47:10.452563 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:47:10.453459 (+ 896us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:47:10.453467 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:47:10.453850 (+ 383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:47:10.453851 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:47:10.453964 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:47:10.454406 (+ 442us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 13:47:10.454414 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0924 13:47:10.455704 (+ 1290us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 13:47:10.457274 (+ 1570us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 13:47:10.457281 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 13:47:10.457283 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 13:47:10.457501 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 13:47:10.457694 (+ 193us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 13:47:10.457695 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 13:47:10.457695 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 13:47:10.457781 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 13:47:10.458003 (+ 222us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 13:47:10.458005 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 13:47:10.458119 (+ 114us) client_negotiation.cc:769] Sending connection context
0924 13:47:10.458194 (+ 75us) client_negotiation.cc:240] Negotiation successful
0924 13:47:10.458332 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":68,"threads_started":1}
I20260924 13:47:10.459106 14156 heartbeater.cc:347] Connected to a master server at 127.13.149.254:37959
I20260924 13:47:10.459213 14156 heartbeater.cc:464] Registering TS with master...
I20260924 13:47:10.459416 14156 heartbeater.cc:511] Master 127.13.149.254:37959 requested a full tablet report, sending...
I20260924 13:47:10.459916 13964 ts_manager.cc:194] Registered new tserver with Master: 3f3e6a719eaa4fac95cedbe8986edc49 (127.13.149.193:45581)
I20260924 13:47:10.460529 13964 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.149.193@KRBTEST.COM'} at 127.13.149.193:40055
WARNING: no policy specified for kudu/127.13.149.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.149.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.149.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.194.keytab.
Entry for principal kudu/127.13.149.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.194.keytab.
I20260924 13:47:10.480007 13911 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJN9WRF/build/release/bin/kudu
/tmp/dist-test-taskJN9WRF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790257629041645-13911-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790257629041645-13911-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790257629041645-13911-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790257629041645-13911-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.149.194:0
--local_ip_for_outbound_sockets=127.13.149.194
--webserver_interface=127.13.149.194
--webserver_port=0
--tserver_master_addrs=127.13.149.254:37959
--keytab_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.194.keytab
--principal=kudu/127.13.149.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.149.212:40499
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790257629041645-13911-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790257629041645-13911-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790257629041645-13911-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 13:47:10.559406 14161 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 13:47:10.559609 14161 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 13:47:10.559633 14161 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 13:47:10.560734 14161 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 13:47:10.560768 14161 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 13:47:10.560796 14161 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.149.194
I20260924 13:47:10.562203 14161 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.149.212:40499
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790257629041645-13911-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790257629041645-13911-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.194.keytab
--principal=kudu/127.13.149.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.149.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790257629041645-13911-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.149.194
--webserver_port=0
--tserver_master_addrs=127.13.149.254:37959
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14161
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.149.194
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790257629041645-13911-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 12:29:37 UTC on ccycloud.quasar-iqzbxn.root.comops.site
build id 4670106
I20260924 13:47:10.562392 14161 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 13:47:10.562565 14161 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 13:47:10.567353 14166 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 13:47:10.567502 14161 server_base.cc:1034] running on GCE node
W20260924 13:47:10.567366 14167 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 13:47:10.567363 14169 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 13:47:10.567830 14161 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 13:47:10.568055 14161 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 13:47:10.569200 14161 hybrid_clock.cc:648] HybridClock initialized: now 1790257630569165 us; error 49 us; skew 500 ppm
Sep 24 13:47:10 dist-test-slave-sjl2 krb5kdc[13918](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257630, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.149.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 13:47:10.571511 14161 init.cc:377] Logged in from keytab as kudu/127.13.149.194@KRBTEST.COM (short username kudu)
I20260924 13:47:10.571965 14161 webserver.cc:460] Webserver started at http://127.13.149.194:38227/ using document root <none> and password file <none>
I20260924 13:47:10.572178 14161 fs_manager.cc:359] Metadata directory not provided
I20260924 13:47:10.572225 14161 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 13:47:10.572348 14161 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 13:47:10.573437 14161 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790257629041645-13911-0/minicluster-data/ts-1/data/instance:
uuid: "4d99858279584ceb82f96e3953b17dab"
format_stamp: "Formatted at 2026-09-24 13:47:10 on dist-test-slave-sjl2"
server_key: "26f404798e644474bf20ea4b66c1fb96"
server_key_iv: "a6aeede10149a23e2b89f3aab52a62d1"
server_key_version: "encryptionkey@0"
I20260924 13:47:10.573822 14161 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790257629041645-13911-0/minicluster-data/ts-1/wal/instance:
uuid: "4d99858279584ceb82f96e3953b17dab"
format_stamp: "Formatted at 2026-09-24 13:47:10 on dist-test-slave-sjl2"
server_key: "26f404798e644474bf20ea4b66c1fb96"
server_key_iv: "a6aeede10149a23e2b89f3aab52a62d1"
server_key_version: "encryptionkey@0"
I20260924 13:47:10.575037 14161 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260924 13:47:10.575682 14176 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 13:47:10.575846 14161 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 13:47:10.575920 14161 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790257629041645-13911-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790257629041645-13911-0/minicluster-data/ts-1/wal
uuid: "4d99858279584ceb82f96e3953b17dab"
format_stamp: "Formatted at 2026-09-24 13:47:10 on dist-test-slave-sjl2"
server_key: "26f404798e644474bf20ea4b66c1fb96"
server_key_iv: "a6aeede10149a23e2b89f3aab52a62d1"
server_key_version: "encryptionkey@0"
I20260924 13:47:10.575984 14161 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790257629041645-13911-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790257629041645-13911-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790257629041645-13911-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 13:47:10.587967 14161 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 13:47:10.589994 14161 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 13:47:10.590135 14161 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 13:47:10.590361 14161 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 13:47:10.590660 14161 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 13:47:10.590694 14161 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 13:47:10.590723 14161 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 13:47:10.590741 14161 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 13:47:10.596369 14161 rpc_server.cc:307] RPC server started. Bound to: 127.13.149.194:41777
I20260924 13:47:10.596417 14290 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.149.194:41777 every 8 connection(s)
I20260924 13:47:10.596741 14161 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790257629041645-13911-0/minicluster-data/ts-1/data/info.pb
Sep 24 13:47:10 dist-test-slave-sjl2 krb5kdc[13918](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257630, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.149.194@KRBTEST.COM for kudu/127.13.149.254@KRBTEST.COM
I20260924 13:47:10.604516 13911 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJN9WRF/build/release/bin/kudu as pid 14161
I20260924 13:47:10.604607 13911 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790257629041645-13911-0/minicluster-data/ts-1/wal/instance
I20260924 13:47:10.604792 13911 external_mini_cluster.cc:1428] Setting key 0cde2e53a44e6e5e950ac0614cebd1bc
I20260924 13:47:10.605845 14015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:10.597466 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.149.194:51123 (local address 127.13.149.254:37959)
0924 13:47:10.597533 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0924 13:47:10.597535 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:47:10.597716 (+ 181us) server_negotiation.cc:407] Connection header received
0924 13:47:10.598239 (+ 523us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:47:10.598240 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:47:10.598254 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:47:10.598305 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:47:10.599265 (+ 960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:10.599801 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:10.600817 (+ 1016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:10.601050 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:10.601105 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:47:10.603180 (+ 2075us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:47:10.603182 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:47:10.603182 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 13:47:10.603191 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:47:10.604357 (+ 1166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:47:10.604827 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:47:10.604828 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:47:10.604828 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:47:10.604894 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:47:10.605171 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:47:10.605171 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:47:10.605172 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:47:10.605291 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:47:10.605350 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0924 13:47:10.605669 (+ 319us) server_negotiation.cc:299] Negotiation successful
0924 13:47:10.605756 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260924 13:47:10.606036 14293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:10.597449 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:37959 (local address 127.13.149.194:51123)
0924 13:47:10.597599 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0924 13:47:10.597636 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0924 13:47:10.598133 (+ 497us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:47:10.598383 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:47:10.598387 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:47:10.598963 (+ 576us) client_negotiation.cc:189] Negotiated authn=SASL
0924 13:47:10.599163 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:47:10.599169 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:47:10.599887 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:47:10.599888 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:47:10.600732 (+ 844us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:47:10.600734 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:47:10.601129 (+ 395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:47:10.601130 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:47:10.601249 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:47:10.601742 (+ 493us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 13:47:10.601750 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0924 13:47:10.603069 (+ 1319us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 13:47:10.604450 (+ 1381us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 13:47:10.604459 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 13:47:10.604461 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 13:47:10.604733 (+ 272us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 13:47:10.604981 (+ 248us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 13:47:10.604982 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 13:47:10.604983 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 13:47:10.605121 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 13:47:10.605345 (+ 224us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 13:47:10.605352 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 13:47:10.605531 (+ 179us) client_negotiation.cc:769] Sending connection context
0924 13:47:10.605643 (+ 112us) client_negotiation.cc:240] Negotiation successful
0924 13:47:10.605808 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":69,"threads_started":1}
I20260924 13:47:10.606591 14291 heartbeater.cc:347] Connected to a master server at 127.13.149.254:37959
I20260924 13:47:10.606706 14291 heartbeater.cc:464] Registering TS with master...
I20260924 13:47:10.606920 14291 heartbeater.cc:511] Master 127.13.149.254:37959 requested a full tablet report, sending...
I20260924 13:47:10.607329 13964 ts_manager.cc:194] Registered new tserver with Master: 4d99858279584ceb82f96e3953b17dab (127.13.149.194:41777)
I20260924 13:47:10.607693 13964 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.149.194@KRBTEST.COM'} at 127.13.149.194:51123
WARNING: no policy specified for kudu/127.13.149.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.149.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.149.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.195.keytab.
Entry for principal kudu/127.13.149.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.195.keytab.
I20260924 13:47:10.629264 13911 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJN9WRF/build/release/bin/kudu
/tmp/dist-test-taskJN9WRF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790257629041645-13911-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790257629041645-13911-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790257629041645-13911-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790257629041645-13911-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.149.195:0
--local_ip_for_outbound_sockets=127.13.149.195
--webserver_interface=127.13.149.195
--webserver_port=0
--tserver_master_addrs=127.13.149.254:37959
--keytab_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.195.keytab
--principal=kudu/127.13.149.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.149.212:40499
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790257629041645-13911-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790257629041645-13911-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790257629041645-13911-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 13:47:10.720521 14296 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 13:47:10.720731 14296 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 13:47:10.720754 14296 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 13:47:10.721984 14296 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 13:47:10.722028 14296 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 13:47:10.722057 14296 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.149.195
I20260924 13:47:10.723706 14296 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.149.212:40499
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790257629041645-13911-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790257629041645-13911-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.195.keytab
--principal=kudu/127.13.149.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.149.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790257629041645-13911-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.149.195
--webserver_port=0
--tserver_master_addrs=127.13.149.254:37959
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14296
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.149.195
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790257629041645-13911-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 12:29:37 UTC on ccycloud.quasar-iqzbxn.root.comops.site
build id 4670106
I20260924 13:47:10.723937 14296 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 13:47:10.724140 14296 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 13:47:10.729130 14301 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 13:47:10.729147 14302 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 13:47:10.729175 14296 server_base.cc:1034] running on GCE node
W20260924 13:47:10.729133 14304 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 13:47:10.729475 14296 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 13:47:10.729679 14296 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 13:47:10.730806 14296 hybrid_clock.cc:648] HybridClock initialized: now 1790257630730795 us; error 30 us; skew 500 ppm
Sep 24 13:47:10 dist-test-slave-sjl2 krb5kdc[13918](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257630, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.149.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 13:47:10.733214 14296 init.cc:377] Logged in from keytab as kudu/127.13.149.195@KRBTEST.COM (short username kudu)
I20260924 13:47:10.733692 14296 webserver.cc:460] Webserver started at http://127.13.149.195:46871/ using document root <none> and password file <none>
I20260924 13:47:10.733897 14296 fs_manager.cc:359] Metadata directory not provided
I20260924 13:47:10.733939 14296 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 13:47:10.734052 14296 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 13:47:10.735100 14296 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790257629041645-13911-0/minicluster-data/ts-2/data/instance:
uuid: "597cc45c024f400bb10d902d0849c2eb"
format_stamp: "Formatted at 2026-09-24 13:47:10 on dist-test-slave-sjl2"
server_key: "fa7f5adcfa8d69e2537003c8459706f7"
server_key_iv: "eb92f1c558419bb26e3922643aa0a246"
server_key_version: "encryptionkey@0"
I20260924 13:47:10.735419 14296 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790257629041645-13911-0/minicluster-data/ts-2/wal/instance:
uuid: "597cc45c024f400bb10d902d0849c2eb"
format_stamp: "Formatted at 2026-09-24 13:47:10 on dist-test-slave-sjl2"
server_key: "fa7f5adcfa8d69e2537003c8459706f7"
server_key_iv: "eb92f1c558419bb26e3922643aa0a246"
server_key_version: "encryptionkey@0"
I20260924 13:47:10.736626 14296 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260924 13:47:10.737349 14311 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 13:47:10.737500 14296 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260924 13:47:10.737583 14296 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790257629041645-13911-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790257629041645-13911-0/minicluster-data/ts-2/wal
uuid: "597cc45c024f400bb10d902d0849c2eb"
format_stamp: "Formatted at 2026-09-24 13:47:10 on dist-test-slave-sjl2"
server_key: "fa7f5adcfa8d69e2537003c8459706f7"
server_key_iv: "eb92f1c558419bb26e3922643aa0a246"
server_key_version: "encryptionkey@0"
I20260924 13:47:10.737636 14296 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790257629041645-13911-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790257629041645-13911-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790257629041645-13911-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 13:47:10.755915 14296 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 13:47:10.758023 14296 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 13:47:10.758160 14296 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 13:47:10.758355 14296 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 13:47:10.758647 14296 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 13:47:10.758684 14296 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 13:47:10.758708 14296 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 13:47:10.758718 14296 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 13:47:10.764135 14296 rpc_server.cc:307] RPC server started. Bound to: 127.13.149.195:38897
I20260924 13:47:10.764199 14425 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.149.195:38897 every 8 connection(s)
I20260924 13:47:10.764539 14296 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790257629041645-13911-0/minicluster-data/ts-2/data/info.pb
Sep 24 13:47:10 dist-test-slave-sjl2 krb5kdc[13918](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257630, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.149.195@KRBTEST.COM for kudu/127.13.149.254@KRBTEST.COM
I20260924 13:47:10.773283 14015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:10.765266 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.149.195:44731 (local address 127.13.149.254:37959)
0924 13:47:10.765320 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0924 13:47:10.765322 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:47:10.765522 (+ 200us) server_negotiation.cc:407] Connection header received
0924 13:47:10.766446 (+ 924us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:47:10.766447 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:47:10.766457 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:47:10.766515 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:47:10.767159 (+ 644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:10.767655 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:10.768619 (+ 964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:10.768796 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:10.768840 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:47:10.770932 (+ 2092us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:47:10.770933 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:47:10.770934 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 13:47:10.770941 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:47:10.772167 (+ 1226us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:47:10.772509 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:47:10.772509 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:47:10.772510 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:47:10.772554 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:47:10.772723 (+ 169us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:47:10.772724 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:47:10.772724 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:47:10.772848 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:47:10.772893 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0924 13:47:10.773155 (+ 262us) server_negotiation.cc:299] Negotiation successful
0924 13:47:10.773215 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260924 13:47:10.773473 14428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:10.765246 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:37959 (local address 127.13.149.195:44731)
0924 13:47:10.765405 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0924 13:47:10.765446 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0924 13:47:10.766355 (+ 909us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:47:10.766573 (+ 218us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:47:10.766576 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:47:10.766864 (+ 288us) client_negotiation.cc:189] Negotiated authn=SASL
0924 13:47:10.767069 (+ 205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:47:10.767074 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:47:10.767724 (+ 650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:47:10.767724 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:47:10.768554 (+ 830us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:47:10.768557 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:47:10.768854 (+ 297us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:47:10.768855 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:47:10.769002 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:47:10.769494 (+ 492us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 13:47:10.769503 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0924 13:47:10.770844 (+ 1341us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 13:47:10.772235 (+ 1391us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 13:47:10.772240 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 13:47:10.772242 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 13:47:10.772454 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 13:47:10.772598 (+ 144us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 13:47:10.772598 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 13:47:10.772598 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 13:47:10.772684 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 13:47:10.772890 (+ 206us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 13:47:10.772893 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 13:47:10.773051 (+ 158us) client_negotiation.cc:769] Sending connection context
0924 13:47:10.773133 (+ 82us) client_negotiation.cc:240] Negotiation successful
0924 13:47:10.773250 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":78,"threads_started":1}
I20260924 13:47:10.773759 13911 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJN9WRF/build/release/bin/kudu as pid 14296
I20260924 13:47:10.773841 13911 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790257629041645-13911-0/minicluster-data/ts-2/wal/instance
I20260924 13:47:10.774008 13911 external_mini_cluster.cc:1428] Setting key d05570f6d0a743c8795a29e26fbd2cdd
I20260924 13:47:10.774032 14426 heartbeater.cc:347] Connected to a master server at 127.13.149.254:37959
I20260924 13:47:10.774132 14426 heartbeater.cc:464] Registering TS with master...
I20260924 13:47:10.774302 14426 heartbeater.cc:511] Master 127.13.149.254:37959 requested a full tablet report, sending...
I20260924 13:47:10.774637 13964 ts_manager.cc:194] Registered new tserver with Master: 597cc45c024f400bb10d902d0849c2eb (127.13.149.195:38897)
I20260924 13:47:10.775122 13964 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.149.195@KRBTEST.COM'} at 127.13.149.195:44731
I20260924 13:47:10.775172 13911 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 24 13:47:10 dist-test-slave-sjl2 krb5kdc[13918](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257630, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 24 13:47:10 dist-test-slave-sjl2 krb5kdc[13918](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257630, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.13.149.193@KRBTEST.COM
I20260924 13:47:10.796746 14433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:10.789172 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47500 (local address 127.13.149.193:45581)
0924 13:47:10.789361 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0924 13:47:10.789366 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 13:47:10.789377 (+ 11us) server_negotiation.cc:407] Connection header received
0924 13:47:10.789404 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:47:10.789407 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:47:10.789420 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:47:10.789482 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:47:10.790060 (+ 578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:10.790795 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:10.791492 (+ 697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:10.791819 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:10.791881 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:47:10.793960 (+ 2079us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:47:10.793968 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:47:10.793969 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 13:47:10.793978 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:47:10.795481 (+ 1503us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:47:10.795881 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:47:10.795885 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:47:10.795885 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:47:10.795952 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:47:10.796144 (+ 192us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:47:10.796145 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:47:10.796146 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:47:10.796339 (+ 193us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:47:10.796482 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0924 13:47:10.796559 (+ 77us) server_negotiation.cc:299] Negotiation successful
0924 13:47:10.796638 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":150,"thread_start_us":89,"threads_started":1}
W20260924 13:47:10.797343 14069 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47500
Sep 24 13:47:10 dist-test-slave-sjl2 krb5kdc[13918](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257630, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 24 13:47:10 dist-test-slave-sjl2 krb5kdc[13918](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257630, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.149.193@KRBTEST.COM
I20260924 13:47:10.817616 14433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:10.811370 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47504 (local address 127.13.149.193:45581)
0924 13:47:10.811436 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0924 13:47:10.811438 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:47:10.811504 (+ 66us) server_negotiation.cc:407] Connection header received
0924 13:47:10.811573 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:47:10.811574 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:47:10.811587 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:47:10.811634 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:47:10.812145 (+ 511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:10.812633 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:10.813346 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:10.813550 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:10.813606 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:47:10.815678 (+ 2072us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:47:10.815680 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:47:10.815680 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 13:47:10.815687 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:47:10.816798 (+ 1111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:47:10.817103 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:47:10.817104 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:47:10.817104 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:47:10.817147 (+ 43us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:47:10.817278 (+ 131us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:47:10.817278 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:47:10.817279 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:47:10.817357 (+ 78us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:47:10.817440 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0924 13:47:10.817495 (+ 55us) server_negotiation.cc:299] Negotiation successful
0924 13:47:10.817551 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260924 13:47:10.818111 13911 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJN9WRF/build/release/bin/kudu with pid 14026
I20260924 13:47:10.822391 13911 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJN9WRF/build/release/bin/kudu with pid 14161
I20260924 13:47:10.827031 13911 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJN9WRF/build/release/bin/kudu with pid 14296
I20260924 13:47:10.832696 13911 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJN9WRF/build/release/bin/kudu with pid 13931
2026-09-24T13:47:10Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (1798 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790257629041645-13911-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 13:47:10 dist-test-slave-sjl2 krb5kdc[14440](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 13:47:10 dist-test-slave-sjl2 krb5kdc[14440](info): set up 2 sockets
Sep 24 13:47:10 dist-test-slave-sjl2 krb5kdc[14440](info): commencing operation
krb5kdc: starting...
W20260924 13:47:11.789476 13911 mini_kdc.cc:121] Time spent starting KDC: real 0.938s user 0.003s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790257629041645-13911-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790257629041645-13911-0/krb5kdc/test-user.keytab.
Sep 24 13:47:11 dist-test-slave-sjl2 krb5kdc[14440](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257631, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T13:47:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T13:47:11Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.149.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.149.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.149.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.254.keytab.
Entry for principal kudu/127.13.149.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.254.keytab.
I20260924 13:47:11.905704 13911 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJN9WRF/build/release/bin/kudu
/tmp/dist-test-taskJN9WRF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790257629041645-13911-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790257629041645-13911-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790257629041645-13911-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790257629041645-13911-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.149.254:42221
--webserver_interface=127.13.149.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.254.keytab
--principal=kudu/127.13.149.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.149.212:33455
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.149.254:42221
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790257629041645-13911-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790257629041645-13911-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790257629041645-13911-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 13:47:11.984575 14453 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 13:47:11.984791 14453 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 13:47:11.984812 14453 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 13:47:11.986017 14453 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 13:47:11.986059 14453 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 13:47:11.986073 14453 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 13:47:11.986084 14453 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 13:47:11.986105 14453 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 13:47:11.987605 14453 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.149.212:33455
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790257629041645-13911-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790257629041645-13911-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.149.254:42221
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.254.keytab
--principal=kudu/127.13.149.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.149.254:42221
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790257629041645-13911-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.149.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14453
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790257629041645-13911-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 12:29:37 UTC on ccycloud.quasar-iqzbxn.root.comops.site
build id 4670106
I20260924 13:47:11.987838 14453 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 13:47:11.988049 14453 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 13:47:11.992668 14458 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 13:47:11.992676 14461 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 13:47:11.992694 14459 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 13:47:11.992901 14453 server_base.cc:1034] running on GCE node
I20260924 13:47:11.993161 14453 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 13:47:11.993433 14453 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 13:47:11.994585 14453 hybrid_clock.cc:648] HybridClock initialized: now 1790257631994571 us; error 26 us; skew 500 ppm
Sep 24 13:47:11 dist-test-slave-sjl2 krb5kdc[14440](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257631, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.149.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 13:47:11.996855 14453 init.cc:377] Logged in from keytab as kudu/127.13.149.254@KRBTEST.COM (short username kudu)
I20260924 13:47:11.997313 14453 webserver.cc:460] Webserver started at http://127.13.149.254:40773/ using document root <none> and password file <none>
I20260924 13:47:11.997524 14453 fs_manager.cc:359] Metadata directory not provided
I20260924 13:47:11.997572 14453 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 13:47:11.997692 14453 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 13:47:11.998734 14453 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790257629041645-13911-0/minicluster-data/master-0/data/instance:
uuid: "cd101688c65543db86c438705480a593"
format_stamp: "Formatted at 2026-09-24 13:47:11 on dist-test-slave-sjl2"
server_key: "6b651eef225634b2f60bafbaedc3fbf0"
server_key_iv: "9f775c4cb6991d31e3f8e71185759d29"
server_key_version: "encryptionkey@0"
I20260924 13:47:11.999042 14453 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790257629041645-13911-0/minicluster-data/master-0/wal/instance:
uuid: "cd101688c65543db86c438705480a593"
format_stamp: "Formatted at 2026-09-24 13:47:11 on dist-test-slave-sjl2"
server_key: "6b651eef225634b2f60bafbaedc3fbf0"
server_key_iv: "9f775c4cb6991d31e3f8e71185759d29"
server_key_version: "encryptionkey@0"
I20260924 13:47:12.000264 14453 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260924 13:47:12.000911 14468 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 13:47:12.001070 14453 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260924 13:47:12.001148 14453 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790257629041645-13911-0/minicluster-data/master-0/data,/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790257629041645-13911-0/minicluster-data/master-0/wal
uuid: "cd101688c65543db86c438705480a593"
format_stamp: "Formatted at 2026-09-24 13:47:11 on dist-test-slave-sjl2"
server_key: "6b651eef225634b2f60bafbaedc3fbf0"
server_key_iv: "9f775c4cb6991d31e3f8e71185759d29"
server_key_version: "encryptionkey@0"
I20260924 13:47:12.001225 14453 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790257629041645-13911-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790257629041645-13911-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790257629041645-13911-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 13:47:12.009263 14453 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 13:47:12.011003 14453 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 13:47:12.011123 14453 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 13:47:12.014961 14453 rpc_server.cc:307] RPC server started. Bound to: 127.13.149.254:42221
I20260924 13:47:12.014989 14521 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.149.254:42221 every 8 connection(s)
I20260924 13:47:12.015334 14453 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790257629041645-13911-0/minicluster-data/master-0/data/info.pb
I20260924 13:47:12.016122 14522 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 13:47:12.018421 14522 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cd101688c65543db86c438705480a593: Bootstrap starting.
I20260924 13:47:12.018950 14522 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cd101688c65543db86c438705480a593: Neither blocks nor log segments found. Creating new log.
I20260924 13:47:12.019198 14522 log.cc:824] T 00000000000000000000000000000000 P cd101688c65543db86c438705480a593: Log is configured to *not* fsync() on all Append() calls
I20260924 13:47:12.019853 14522 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cd101688c65543db86c438705480a593: No bootstrap required, opened a new log
I20260924 13:47:12.019873 13911 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJN9WRF/build/release/bin/kudu as pid 14453
I20260924 13:47:12.019958 13911 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790257629041645-13911-0/minicluster-data/master-0/wal/instance
I20260924 13:47:12.020117 13911 external_mini_cluster.cc:1428] Setting key 414f34c5087c1e98dc218590c7e9d1da
I20260924 13:47:12.021654 14522 raft_consensus.cc:348] T 00000000000000000000000000000000 P cd101688c65543db86c438705480a593 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cd101688c65543db86c438705480a593" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 42221 } }
I20260924 13:47:12.021785 14522 raft_consensus.cc:374] T 00000000000000000000000000000000 P cd101688c65543db86c438705480a593 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 13:47:12.021806 14522 raft_consensus.cc:729] T 00000000000000000000000000000000 P cd101688c65543db86c438705480a593 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd101688c65543db86c438705480a593, State: Initialized, Role: FOLLOWER
I20260924 13:47:12.021901 14522 consensus_queue.cc:260] T 00000000000000000000000000000000 P cd101688c65543db86c438705480a593 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cd101688c65543db86c438705480a593" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 42221 } }
I20260924 13:47:12.021970 14522 raft_consensus.cc:388] T 00000000000000000000000000000000 P cd101688c65543db86c438705480a593 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 13:47:12.021996 14522 raft_consensus.cc:482] T 00000000000000000000000000000000 P cd101688c65543db86c438705480a593 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 13:47:12.022042 14522 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cd101688c65543db86c438705480a593 [term 0 FOLLOWER]: Advancing to term 1
I20260924 13:47:12.022835 14522 raft_consensus.cc:504] T 00000000000000000000000000000000 P cd101688c65543db86c438705480a593 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cd101688c65543db86c438705480a593" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 42221 } }
I20260924 13:47:12.022929 14522 leader_election.cc:302] T 00000000000000000000000000000000 P cd101688c65543db86c438705480a593 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cd101688c65543db86c438705480a593; no voters:
I20260924 13:47:12.023075 14522 leader_election.cc:288] T 00000000000000000000000000000000 P cd101688c65543db86c438705480a593 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 13:47:12.023111 14527 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cd101688c65543db86c438705480a593 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 13:47:12.023206 14527 raft_consensus.cc:686] T 00000000000000000000000000000000 P cd101688c65543db86c438705480a593 [term 1 LEADER]: Becoming Leader. State: Replica: cd101688c65543db86c438705480a593, State: Running, Role: LEADER
I20260924 13:47:12.023313 14522 sys_catalog.cc:564] T 00000000000000000000000000000000 P cd101688c65543db86c438705480a593 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 13:47:12.023285 14527 consensus_queue.cc:237] T 00000000000000000000000000000000 P cd101688c65543db86c438705480a593 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cd101688c65543db86c438705480a593" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 42221 } }
I20260924 13:47:12.023639 14527 sys_catalog.cc:455] T 00000000000000000000000000000000 P cd101688c65543db86c438705480a593 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cd101688c65543db86c438705480a593. Latest consensus state: current_term: 1 leader_uuid: "cd101688c65543db86c438705480a593" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cd101688c65543db86c438705480a593" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 42221 } } }
I20260924 13:47:12.023711 14527 sys_catalog.cc:458] T 00000000000000000000000000000000 P cd101688c65543db86c438705480a593 [sys.catalog]: This master's current role is: LEADER
I20260924 13:47:12.023664 14529 sys_catalog.cc:455] T 00000000000000000000000000000000 P cd101688c65543db86c438705480a593 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cd101688c65543db86c438705480a593" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cd101688c65543db86c438705480a593" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 42221 } } }
I20260924 13:47:12.023881 14529 sys_catalog.cc:458] T 00000000000000000000000000000000 P cd101688c65543db86c438705480a593 [sys.catalog]: This master's current role is: LEADER
I20260924 13:47:12.023940 14533 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 13:47:12.024423 14533 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 13:47:12.026247 14533 catalog_manager.cc:1347] Generated new cluster ID: 9c0d94408d5c448daff3fb160973d4b4
I20260924 13:47:12.026309 14533 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 24 13:47:12 dist-test-slave-sjl2 krb5kdc[14440](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257631, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.149.254@KRBTEST.COM
I20260924 13:47:12.031885 14526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:12.020598 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43792 (local address 127.13.149.254:42221)
0924 13:47:12.020775 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0924 13:47:12.020785 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0924 13:47:12.020813 (+ 28us) server_negotiation.cc:407] Connection header received
0924 13:47:12.021647 (+ 834us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:47:12.021657 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:47:12.021714 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:47:12.021859 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:47:12.022634 (+ 775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:12.023953 (+ 1319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:12.025087 (+ 1134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:12.025539 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:12.025651 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:47:12.028324 (+ 2673us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:47:12.028328 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:47:12.028329 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 13:47:12.028340 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:47:12.030189 (+ 1849us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:47:12.030580 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:47:12.030583 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:47:12.030584 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:47:12.030692 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:47:12.030888 (+ 196us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:47:12.030889 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:47:12.030889 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:47:12.031172 (+ 283us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:47:12.031306 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0924 13:47:12.031420 (+ 114us) server_negotiation.cc:299] Negotiation successful
0924 13:47:12.031647 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":69,"threads_started":1}
I20260924 13:47:12.057448 14533 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 13:47:12.057991 14533 catalog_manager.cc:1504] Loading token signing keys...
I20260924 13:47:12.065071 14533 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cd101688c65543db86c438705480a593: Generated new TSK 0
I20260924 13:47:12.065250 14533 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.13.149.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.149.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.149.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.193.keytab.
Entry for principal kudu/127.13.149.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.193.keytab.
I20260924 13:47:12.092167 13911 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJN9WRF/build/release/bin/kudu
/tmp/dist-test-taskJN9WRF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790257629041645-13911-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790257629041645-13911-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790257629041645-13911-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790257629041645-13911-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.149.193:0
--local_ip_for_outbound_sockets=127.13.149.193
--webserver_interface=127.13.149.193
--webserver_port=0
--tserver_master_addrs=127.13.149.254:42221
--keytab_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.193.keytab
--principal=kudu/127.13.149.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.149.212:33455
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790257629041645-13911-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790257629041645-13911-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790257629041645-13911-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 13:47:12.175371 14548 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 13:47:12.175608 14548 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 13:47:12.175642 14548 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 13:47:12.176887 14548 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 13:47:12.176975 14548 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 13:47:12.177011 14548 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.149.193
I20260924 13:47:12.178674 14548 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.149.212:33455
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790257629041645-13911-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790257629041645-13911-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.193.keytab
--principal=kudu/127.13.149.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.149.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790257629041645-13911-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.149.193
--webserver_port=0
--tserver_master_addrs=127.13.149.254:42221
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14548
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.149.193
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790257629041645-13911-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 12:29:37 UTC on ccycloud.quasar-iqzbxn.root.comops.site
build id 4670106
I20260924 13:47:12.178949 14548 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 13:47:12.179188 14548 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260924 13:47:12.184526 14548 server_base.cc:1034] running on GCE node
W20260924 13:47:12.184485 14554 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 13:47:12.184492 14553 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 13:47:12.184486 14556 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 13:47:12.185007 14548 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 13:47:12.185264 14548 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 13:47:12.186448 14548 hybrid_clock.cc:648] HybridClock initialized: now 1790257632186428 us; error 31 us; skew 500 ppm
Sep 24 13:47:12 dist-test-slave-sjl2 krb5kdc[14440](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257632, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.149.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 13:47:12.189141 14548 init.cc:377] Logged in from keytab as kudu/127.13.149.193@KRBTEST.COM (short username kudu)
I20260924 13:47:12.189630 14548 webserver.cc:460] Webserver started at http://127.13.149.193:37033/ using document root <none> and password file <none>
I20260924 13:47:12.189864 14548 fs_manager.cc:359] Metadata directory not provided
I20260924 13:47:12.189915 14548 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 13:47:12.190045 14548 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 13:47:12.191103 14548 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790257629041645-13911-0/minicluster-data/ts-0/data/instance:
uuid: "9b2cd463cecd4a5ba8144a03f54cbd34"
format_stamp: "Formatted at 2026-09-24 13:47:12 on dist-test-slave-sjl2"
server_key: "f70c36a52d28294e6019409bba83d0bd"
server_key_iv: "dedfb413224ec1dde20a60a60de8705c"
server_key_version: "encryptionkey@0"
I20260924 13:47:12.191473 14548 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790257629041645-13911-0/minicluster-data/ts-0/wal/instance:
uuid: "9b2cd463cecd4a5ba8144a03f54cbd34"
format_stamp: "Formatted at 2026-09-24 13:47:12 on dist-test-slave-sjl2"
server_key: "f70c36a52d28294e6019409bba83d0bd"
server_key_iv: "dedfb413224ec1dde20a60a60de8705c"
server_key_version: "encryptionkey@0"
I20260924 13:47:12.192787 14548 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260924 13:47:12.193692 14563 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 13:47:12.193893 14548 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260924 13:47:12.193981 14548 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790257629041645-13911-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790257629041645-13911-0/minicluster-data/ts-0/wal
uuid: "9b2cd463cecd4a5ba8144a03f54cbd34"
format_stamp: "Formatted at 2026-09-24 13:47:12 on dist-test-slave-sjl2"
server_key: "f70c36a52d28294e6019409bba83d0bd"
server_key_iv: "dedfb413224ec1dde20a60a60de8705c"
server_key_version: "encryptionkey@0"
I20260924 13:47:12.194052 14548 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790257629041645-13911-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790257629041645-13911-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790257629041645-13911-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 13:47:12.216423 14548 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 13:47:12.218751 14548 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 13:47:12.218911 14548 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 13:47:12.219135 14548 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 13:47:12.219440 14548 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 13:47:12.219475 14548 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 13:47:12.219504 14548 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 13:47:12.219522 14548 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 13:47:12.225265 14548 rpc_server.cc:307] RPC server started. Bound to: 127.13.149.193:41825
I20260924 13:47:12.225318 14676 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.149.193:41825 every 8 connection(s)
I20260924 13:47:12.225641 14548 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790257629041645-13911-0/minicluster-data/ts-0/data/info.pb
I20260924 13:47:12.227608 13911 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJN9WRF/build/release/bin/kudu as pid 14548
I20260924 13:47:12.227697 13911 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790257629041645-13911-0/minicluster-data/ts-0/wal/instance
I20260924 13:47:12.227840 13911 external_mini_cluster.cc:1428] Setting key dd261c8f070203644a336ab190a9fa97
Sep 24 13:47:12 dist-test-slave-sjl2 krb5kdc[14440](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257632, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.149.193@KRBTEST.COM for kudu/127.13.149.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.13.149.194@KRBTEST.COM; defaulting to no policy
I20260924 13:47:12.236971 14526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:12.226428 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.149.193:52139 (local address 127.13.149.254:42221)
0924 13:47:12.226484 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0924 13:47:12.226486 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:47:12.226634 (+ 148us) server_negotiation.cc:407] Connection header received
0924 13:47:12.227830 (+ 1196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:47:12.227831 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:47:12.227850 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:47:12.227921 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:47:12.228735 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:12.229505 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:12.231036 (+ 1531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:12.231275 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:12.231359 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:47:12.233999 (+ 2640us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:47:12.234000 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:47:12.234001 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 13:47:12.234009 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:47:12.235343 (+ 1334us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:47:12.235867 (+ 524us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:47:12.235868 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:47:12.235869 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:47:12.235939 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:47:12.236223 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:47:12.236225 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:47:12.236225 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:47:12.236462 (+ 237us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:47:12.236533 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0924 13:47:12.236793 (+ 260us) server_negotiation.cc:299] Negotiation successful
0924 13:47:12.236870 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260924 13:47:12.237195 14679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:12.226348 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:42221 (local address 127.13.149.193:52139)
0924 13:47:12.226512 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0924 13:47:12.226556 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0924 13:47:12.227625 (+ 1069us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:47:12.228011 (+ 386us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:47:12.228016 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:47:12.228387 (+ 371us) client_negotiation.cc:189] Negotiated authn=SASL
0924 13:47:12.228648 (+ 261us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:47:12.228654 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:47:12.229615 (+ 961us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:47:12.229616 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:47:12.230927 (+ 1311us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:47:12.230930 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:47:12.231377 (+ 447us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:47:12.231378 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:47:12.231534 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:47:12.232124 (+ 590us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 13:47:12.232135 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0924 13:47:12.233863 (+ 1728us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 13:47:12.235434 (+ 1571us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 13:47:12.235444 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 13:47:12.235447 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0924 13:47:12.235747 (+ 300us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 13:47:12.236014 (+ 267us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 13:47:12.236015 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 13:47:12.236016 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 13:47:12.236162 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 13:47:12.236536 (+ 374us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 13:47:12.236542 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 13:47:12.236681 (+ 139us) client_negotiation.cc:769] Sending connection context
0924 13:47:12.236802 (+ 121us) client_negotiation.cc:240] Negotiation successful
0924 13:47:12.236965 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":73,"threads_started":1}
I20260924 13:47:12.237674 14677 heartbeater.cc:347] Connected to a master server at 127.13.149.254:42221
I20260924 13:47:12.237782 14677 heartbeater.cc:464] Registering TS with master...
I20260924 13:47:12.237967 14677 heartbeater.cc:511] Master 127.13.149.254:42221 requested a full tablet report, sending...
I20260924 13:47:12.238507 14485 ts_manager.cc:194] Registered new tserver with Master: 9b2cd463cecd4a5ba8144a03f54cbd34 (127.13.149.193:41825)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.149.194@KRBTEST.COM" created.
I20260924 13:47:12.239370 14485 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.149.193@KRBTEST.COM'} at 127.13.149.193:52139
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.149.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.194.keytab.
Entry for principal kudu/127.13.149.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.194.keytab.
I20260924 13:47:12.255959 13911 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJN9WRF/build/release/bin/kudu
/tmp/dist-test-taskJN9WRF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790257629041645-13911-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790257629041645-13911-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790257629041645-13911-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790257629041645-13911-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.149.194:0
--local_ip_for_outbound_sockets=127.13.149.194
--webserver_interface=127.13.149.194
--webserver_port=0
--tserver_master_addrs=127.13.149.254:42221
--keytab_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.194.keytab
--principal=kudu/127.13.149.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.149.212:33455
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790257629041645-13911-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790257629041645-13911-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790257629041645-13911-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 13:47:12.340130 14682 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 13:47:12.340368 14682 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 13:47:12.340394 14682 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 13:47:12.341624 14682 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 13:47:12.341676 14682 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 13:47:12.341704 14682 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.149.194
I20260924 13:47:12.343252 14682 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.149.212:33455
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790257629041645-13911-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790257629041645-13911-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.194.keytab
--principal=kudu/127.13.149.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.149.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790257629041645-13911-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.149.194
--webserver_port=0
--tserver_master_addrs=127.13.149.254:42221
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14682
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.149.194
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790257629041645-13911-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 12:29:37 UTC on ccycloud.quasar-iqzbxn.root.comops.site
build id 4670106
I20260924 13:47:12.343504 14682 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 13:47:12.343698 14682 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 13:47:12.348601 14688 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 13:47:12.348601 14687 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 13:47:12.348603 14691 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 13:47:12.348764 14682 server_base.cc:1034] running on GCE node
I20260924 13:47:12.349118 14682 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 13:47:12.349368 14682 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 13:47:12.350522 14682 hybrid_clock.cc:648] HybridClock initialized: now 1790257632350509 us; error 28 us; skew 500 ppm
Sep 24 13:47:12 dist-test-slave-sjl2 krb5kdc[14440](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257632, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.149.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 13:47:12.353235 14682 init.cc:377] Logged in from keytab as kudu/127.13.149.194@KRBTEST.COM (short username kudu)
I20260924 13:47:12.353737 14682 webserver.cc:460] Webserver started at http://127.13.149.194:43331/ using document root <none> and password file <none>
I20260924 13:47:12.353945 14682 fs_manager.cc:359] Metadata directory not provided
I20260924 13:47:12.353995 14682 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 13:47:12.354122 14682 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 13:47:12.355190 14682 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790257629041645-13911-0/minicluster-data/ts-1/data/instance:
uuid: "70e1bf096f5746f8adcc59a443d9e152"
format_stamp: "Formatted at 2026-09-24 13:47:12 on dist-test-slave-sjl2"
server_key: "d6db19fc84153efb0f5547859dc2b399"
server_key_iv: "872165046568b666d20d190ed953654c"
server_key_version: "encryptionkey@0"
I20260924 13:47:12.355530 14682 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790257629041645-13911-0/minicluster-data/ts-1/wal/instance:
uuid: "70e1bf096f5746f8adcc59a443d9e152"
format_stamp: "Formatted at 2026-09-24 13:47:12 on dist-test-slave-sjl2"
server_key: "d6db19fc84153efb0f5547859dc2b399"
server_key_iv: "872165046568b666d20d190ed953654c"
server_key_version: "encryptionkey@0"
I20260924 13:47:12.356798 14682 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260924 13:47:12.357573 14698 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 13:47:12.357746 14682 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 13:47:12.357828 14682 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790257629041645-13911-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790257629041645-13911-0/minicluster-data/ts-1/wal
uuid: "70e1bf096f5746f8adcc59a443d9e152"
format_stamp: "Formatted at 2026-09-24 13:47:12 on dist-test-slave-sjl2"
server_key: "d6db19fc84153efb0f5547859dc2b399"
server_key_iv: "872165046568b666d20d190ed953654c"
server_key_version: "encryptionkey@0"
I20260924 13:47:12.357889 14682 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790257629041645-13911-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790257629041645-13911-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790257629041645-13911-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 13:47:12.366381 14682 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 13:47:12.368439 14682 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 13:47:12.368580 14682 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 13:47:12.368793 14682 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 13:47:12.369131 14682 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 13:47:12.369174 14682 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 13:47:12.369205 14682 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 13:47:12.369230 14682 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 13:47:12.375025 14682 rpc_server.cc:307] RPC server started. Bound to: 127.13.149.194:46527
I20260924 13:47:12.375072 14814 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.149.194:46527 every 8 connection(s)
I20260924 13:47:12.375394 14682 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790257629041645-13911-0/minicluster-data/ts-1/data/info.pb
I20260924 13:47:12.381057 13911 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJN9WRF/build/release/bin/kudu as pid 14682
I20260924 13:47:12.381172 13911 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790257629041645-13911-0/minicluster-data/ts-1/wal/instance
I20260924 13:47:12.381361 13911 external_mini_cluster.cc:1428] Setting key fcf133d6ae3f14d1257f6dafb7e899b3
Sep 24 13:47:12 dist-test-slave-sjl2 krb5kdc[14440](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257632, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.149.194@KRBTEST.COM for kudu/127.13.149.254@KRBTEST.COM
I20260924 13:47:12.385851 14526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:12.376074 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.149.194:57869 (local address 127.13.149.254:42221)
0924 13:47:12.376121 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0924 13:47:12.376123 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:47:12.376320 (+ 197us) server_negotiation.cc:407] Connection header received
0924 13:47:12.377358 (+ 1038us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:47:12.377359 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:47:12.377373 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:47:12.377422 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:47:12.378079 (+ 657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:12.378580 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:12.379599 (+ 1019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:12.379791 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:12.379857 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:47:12.382621 (+ 2764us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:47:12.382623 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:47:12.382624 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 13:47:12.382631 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:47:12.384205 (+ 1574us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:47:12.384766 (+ 561us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:47:12.384768 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:47:12.384768 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:47:12.384842 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:47:12.385135 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:47:12.385136 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:47:12.385136 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:47:12.385286 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:47:12.385355 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0924 13:47:12.385650 (+ 295us) server_negotiation.cc:299] Negotiation successful
0924 13:47:12.385750 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260924 13:47:12.386039 14817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:12.376039 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:42221 (local address 127.13.149.194:57869)
0924 13:47:12.376189 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0924 13:47:12.376232 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0924 13:47:12.377233 (+ 1001us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:47:12.377499 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:47:12.377504 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:47:12.377783 (+ 279us) client_negotiation.cc:189] Negotiated authn=SASL
0924 13:47:12.377987 (+ 204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:47:12.377992 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:47:12.378665 (+ 673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:47:12.378666 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:47:12.379509 (+ 843us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:47:12.379512 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:47:12.379873 (+ 361us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:47:12.379874 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:47:12.380008 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:47:12.380579 (+ 571us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 13:47:12.380592 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0924 13:47:12.382495 (+ 1903us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 13:47:12.384317 (+ 1822us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 13:47:12.384327 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 13:47:12.384329 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 13:47:12.384582 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 13:47:12.384908 (+ 326us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 13:47:12.384909 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 13:47:12.384909 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 13:47:12.385071 (+ 162us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 13:47:12.385351 (+ 280us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 13:47:12.385359 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 13:47:12.385512 (+ 153us) client_negotiation.cc:769] Sending connection context
0924 13:47:12.385604 (+ 92us) client_negotiation.cc:240] Negotiation successful
0924 13:47:12.385794 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":66,"threads_started":1}
I20260924 13:47:12.386533 14815 heartbeater.cc:347] Connected to a master server at 127.13.149.254:42221
I20260924 13:47:12.386633 14815 heartbeater.cc:464] Registering TS with master...
I20260924 13:47:12.386827 14815 heartbeater.cc:511] Master 127.13.149.254:42221 requested a full tablet report, sending...
I20260924 13:47:12.387218 14485 ts_manager.cc:194] Registered new tserver with Master: 70e1bf096f5746f8adcc59a443d9e152 (127.13.149.194:46527)
I20260924 13:47:12.387609 14485 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.149.194@KRBTEST.COM'} at 127.13.149.194:57869
WARNING: no policy specified for kudu/127.13.149.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.149.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.149.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.195.keytab.
Entry for principal kudu/127.13.149.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.195.keytab.
I20260924 13:47:12.408416 13911 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJN9WRF/build/release/bin/kudu
/tmp/dist-test-taskJN9WRF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790257629041645-13911-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790257629041645-13911-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790257629041645-13911-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790257629041645-13911-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.149.195:0
--local_ip_for_outbound_sockets=127.13.149.195
--webserver_interface=127.13.149.195
--webserver_port=0
--tserver_master_addrs=127.13.149.254:42221
--keytab_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.195.keytab
--principal=kudu/127.13.149.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.149.212:33455
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790257629041645-13911-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790257629041645-13911-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790257629041645-13911-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 13:47:12.490233 14820 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 13:47:12.490491 14820 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 13:47:12.490520 14820 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 13:47:12.491914 14820 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 13:47:12.491982 14820 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 13:47:12.492022 14820 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.149.195
I20260924 13:47:12.493816 14820 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.149.212:33455
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790257629041645-13911-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790257629041645-13911-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.195.keytab
--principal=kudu/127.13.149.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.149.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790257629041645-13911-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.149.195
--webserver_port=0
--tserver_master_addrs=127.13.149.254:42221
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14820
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.149.195
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790257629041645-13911-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 12:29:37 UTC on ccycloud.quasar-iqzbxn.root.comops.site
build id 4670106
I20260924 13:47:12.494086 14820 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 13:47:12.494319 14820 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 13:47:12.499352 14826 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 13:47:12.499352 14825 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 13:47:12.499413 14828 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 13:47:12.499606 14820 server_base.cc:1034] running on GCE node
I20260924 13:47:12.499832 14820 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 13:47:12.500061 14820 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 13:47:12.501199 14820 hybrid_clock.cc:648] HybridClock initialized: now 1790257632501185 us; error 31 us; skew 500 ppm
Sep 24 13:47:12 dist-test-slave-sjl2 krb5kdc[14440](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257632, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.149.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 13:47:12.503708 14820 init.cc:377] Logged in from keytab as kudu/127.13.149.195@KRBTEST.COM (short username kudu)
I20260924 13:47:12.504215 14820 webserver.cc:460] Webserver started at http://127.13.149.195:38127/ using document root <none> and password file <none>
I20260924 13:47:12.504469 14820 fs_manager.cc:359] Metadata directory not provided
I20260924 13:47:12.504521 14820 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 13:47:12.504657 14820 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 13:47:12.506062 14820 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790257629041645-13911-0/minicluster-data/ts-2/data/instance:
uuid: "b417adcdc5bd4fc69acc675f3c57c03d"
format_stamp: "Formatted at 2026-09-24 13:47:12 on dist-test-slave-sjl2"
server_key: "f15ebd0daa3429850acce0fcf09151b1"
server_key_iv: "009cf6b36e7060a9eb70cdd0fbd254d8"
server_key_version: "encryptionkey@0"
I20260924 13:47:12.506425 14820 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790257629041645-13911-0/minicluster-data/ts-2/wal/instance:
uuid: "b417adcdc5bd4fc69acc675f3c57c03d"
format_stamp: "Formatted at 2026-09-24 13:47:12 on dist-test-slave-sjl2"
server_key: "f15ebd0daa3429850acce0fcf09151b1"
server_key_iv: "009cf6b36e7060a9eb70cdd0fbd254d8"
server_key_version: "encryptionkey@0"
I20260924 13:47:12.507722 14820 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260924 13:47:12.508448 14836 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 13:47:12.508594 14820 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260924 13:47:12.508680 14820 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790257629041645-13911-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790257629041645-13911-0/minicluster-data/ts-2/wal
uuid: "b417adcdc5bd4fc69acc675f3c57c03d"
format_stamp: "Formatted at 2026-09-24 13:47:12 on dist-test-slave-sjl2"
server_key: "f15ebd0daa3429850acce0fcf09151b1"
server_key_iv: "009cf6b36e7060a9eb70cdd0fbd254d8"
server_key_version: "encryptionkey@0"
I20260924 13:47:12.508752 14820 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790257629041645-13911-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790257629041645-13911-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790257629041645-13911-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 13:47:12.533891 14820 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 13:47:12.536069 14820 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 13:47:12.536216 14820 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 13:47:12.536438 14820 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 13:47:12.536746 14820 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 13:47:12.536780 14820 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 13:47:12.536809 14820 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 13:47:12.536835 14820 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 13:47:12.542644 14820 rpc_server.cc:307] RPC server started. Bound to: 127.13.149.195:45965
I20260924 13:47:12.542683 14950 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.149.195:45965 every 8 connection(s)
I20260924 13:47:12.543066 14820 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790257629041645-13911-0/minicluster-data/ts-2/data/info.pb
I20260924 13:47:12.543533 13911 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJN9WRF/build/release/bin/kudu as pid 14820
I20260924 13:47:12.543620 13911 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790257629041645-13911-0/minicluster-data/ts-2/wal/instance
I20260924 13:47:12.543769 13911 external_mini_cluster.cc:1428] Setting key db749727801e03af20e6cad6dabb7b9b
Sep 24 13:47:12 dist-test-slave-sjl2 krb5kdc[14440](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257632, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.149.195@KRBTEST.COM for kudu/127.13.149.254@KRBTEST.COM
I20260924 13:47:12.554402 14526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:12.543890 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.149.195:44227 (local address 127.13.149.254:42221)
0924 13:47:12.543949 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0924 13:47:12.543952 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 13:47:12.544352 (+ 400us) server_negotiation.cc:407] Connection header received
0924 13:47:12.545375 (+ 1023us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:47:12.545375 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:47:12.545388 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:47:12.545455 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:47:12.546203 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:12.546886 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:12.548493 (+ 1607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:12.548715 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:12.548795 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:47:12.551345 (+ 2550us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:47:12.551347 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:47:12.551347 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 13:47:12.551356 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:47:12.552770 (+ 1414us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:47:12.553308 (+ 538us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:47:12.553309 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:47:12.553310 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:47:12.553370 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:47:12.553640 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:47:12.553640 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:47:12.553641 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:47:12.553760 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:47:12.553818 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0924 13:47:12.554239 (+ 421us) server_negotiation.cc:299] Negotiation successful
0924 13:47:12.554331 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260924 13:47:12.554525 14953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:12.543885 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:42221 (local address 127.13.149.195:44227)
0924 13:47:12.544152 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0924 13:47:12.544202 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0924 13:47:12.545268 (+ 1066us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:47:12.545521 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:47:12.545525 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:47:12.545819 (+ 294us) client_negotiation.cc:189] Negotiated authn=SASL
0924 13:47:12.546039 (+ 220us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:47:12.546044 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:47:12.547017 (+ 973us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:47:12.547019 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:47:12.548330 (+ 1311us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:47:12.548338 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:47:12.548899 (+ 561us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:47:12.548901 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:47:12.549099 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:47:12.549678 (+ 579us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 13:47:12.549688 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0924 13:47:12.551197 (+ 1509us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 13:47:12.552869 (+ 1672us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 13:47:12.552877 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 13:47:12.552880 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0924 13:47:12.553207 (+ 327us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 13:47:12.553446 (+ 239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 13:47:12.553447 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 13:47:12.553447 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 13:47:12.553575 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 13:47:12.553897 (+ 322us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 13:47:12.553904 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 13:47:12.554059 (+ 155us) client_negotiation.cc:769] Sending connection context
0924 13:47:12.554153 (+ 94us) client_negotiation.cc:240] Negotiation successful
0924 13:47:12.554305 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":218,"thread_start_us":88,"threads_started":1}
I20260924 13:47:12.555104 14951 heartbeater.cc:347] Connected to a master server at 127.13.149.254:42221
I20260924 13:47:12.555219 14951 heartbeater.cc:464] Registering TS with master...
I20260924 13:47:12.555447 14951 heartbeater.cc:511] Master 127.13.149.254:42221 requested a full tablet report, sending...
I20260924 13:47:12.555840 14485 ts_manager.cc:194] Registered new tserver with Master: b417adcdc5bd4fc69acc675f3c57c03d (127.13.149.195:45965)
I20260924 13:47:12.556232 14485 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.149.195@KRBTEST.COM'} at 127.13.149.195:44227
I20260924 13:47:12.566937 13911 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260924 13:47:12.574577 14526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:12.568507 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43808 (local address 127.13.149.254:42221)
0924 13:47:12.568580 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0924 13:47:12.568582 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:47:12.568629 (+ 47us) server_negotiation.cc:407] Connection header received
0924 13:47:12.568684 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:47:12.568685 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:47:12.568695 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:47:12.568743 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:47:12.569371 (+ 628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:12.569898 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:12.570639 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:12.570884 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:12.570938 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:47:12.572210 (+ 1272us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:47:12.572212 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:47:12.572212 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 13:47:12.572219 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:47:12.573459 (+ 1240us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:47:12.573879 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:47:12.573880 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:47:12.573880 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:47:12.573943 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:47:12.574125 (+ 182us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:47:12.574125 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:47:12.574126 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:47:12.574281 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:47:12.574355 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0924 13:47:12.574445 (+ 90us) server_negotiation.cc:299] Negotiation successful
0924 13:47:12.574509 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260924 13:47:12.577059 14485 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43808:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 13:47:12.578722 14485 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 13:47:12.585922 14964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:12.581336 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34102 (local address 127.13.149.194:46527)
0924 13:47:12.581744 (+ 408us) server_negotiation.cc:206] Beginning negotiation
0924 13:47:12.581748 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 13:47:12.581756 (+ 8us) server_negotiation.cc:407] Connection header received
0924 13:47:12.581793 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:47:12.581795 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:47:12.581810 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:47:12.581889 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 13:47:12.582097 (+ 208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:12.583240 (+ 1143us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:12.584825 (+ 1585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:12.585585 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:12.585653 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:47:12.585701 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0924 13:47:12.585780 (+ 79us) server_negotiation.cc:299] Negotiation successful
0924 13:47:12.585823 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":373,"thread_start_us":85,"threads_started":1}
I20260924 13:47:12.585963 14962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:12.581219 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.194:46527 (local address 127.0.0.1:34102)
0924 13:47:12.581333 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0924 13:47:12.581375 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0924 13:47:12.581441 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:47:12.581905 (+ 464us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:47:12.581906 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:47:12.581909 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 13:47:12.582005 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:47:12.582007 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:47:12.583354 (+ 1347us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:47:12.583355 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:47:12.584696 (+ 1341us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:47:12.584701 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:47:12.585651 (+ 950us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:47:12.585651 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:47:12.585705 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:47:12.585708 (+ 3us) client_negotiation.cc:769] Sending connection context
0924 13:47:12.585763 (+ 55us) client_negotiation.cc:240] Negotiation successful
0924 13:47:12.585809 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":87,"mutex_wait_us":55,"thread_start_us":59,"threads_started":1}
I20260924 13:47:12.586107 14965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:12.581441 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.195:45965 (local address 127.0.0.1:57694)
0924 13:47:12.581996 (+ 555us) negotiation.cc:107] Waiting for socket to connect
0924 13:47:12.582007 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0924 13:47:12.582051 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:47:12.582316 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:47:12.582317 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:47:12.582318 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 13:47:12.582363 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:47:12.582365 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:47:12.583216 (+ 851us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:47:12.583217 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:47:12.584725 (+ 1508us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:47:12.584726 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:47:12.585530 (+ 804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:47:12.585531 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:47:12.585640 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:47:12.585659 (+ 19us) client_negotiation.cc:769] Sending connection context
0924 13:47:12.585733 (+ 74us) client_negotiation.cc:240] Negotiation successful
0924 13:47:12.585802 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":538,"mutex_wait_us":52,"thread_start_us":63,"threads_started":1}
I20260924 13:47:12.585963 14963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:12.581224 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.193:41825 (local address 127.0.0.1:43396)
0924 13:47:12.581565 (+ 341us) negotiation.cc:107] Waiting for socket to connect
0924 13:47:12.581575 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 13:47:12.581625 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:47:12.581900 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:47:12.581904 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:47:12.581907 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 13:47:12.581991 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:47:12.581993 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:47:12.583394 (+ 1401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:47:12.583395 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:47:12.584427 (+ 1032us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:47:12.584431 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:47:12.585671 (+ 1240us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:47:12.585672 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:47:12.585744 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:47:12.585749 (+ 5us) client_negotiation.cc:769] Sending connection context
0924 13:47:12.585797 (+ 48us) client_negotiation.cc:240] Negotiation successful
0924 13:47:12.585844 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":322,"mutex_wait_us":7,"thread_start_us":109,"threads_started":1}
I20260924 13:47:12.586143 14966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:12.581539 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43396 (local address 127.13.149.193:41825)
0924 13:47:12.581718 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0924 13:47:12.581723 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 13:47:12.581733 (+ 10us) server_negotiation.cc:407] Connection header received
0924 13:47:12.581775 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:47:12.581777 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:47:12.581792 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:47:12.581864 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 13:47:12.582149 (+ 285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:12.583314 (+ 1165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:12.584494 (+ 1180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:12.585484 (+ 990us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:12.585572 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:47:12.585631 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0924 13:47:12.586012 (+ 381us) server_negotiation.cc:299] Negotiation successful
0924 13:47:12.586067 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":134,"thread_start_us":81,"threads_started":1}
I20260924 13:47:12.586771 14967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:12.581542 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57694 (local address 127.13.149.195:45965)
0924 13:47:12.581727 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0924 13:47:12.581730 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 13:47:12.582103 (+ 373us) server_negotiation.cc:407] Connection header received
0924 13:47:12.582146 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:47:12.582149 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:47:12.582164 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:47:12.582227 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 13:47:12.582434 (+ 207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:12.583104 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:12.584824 (+ 1720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:12.585427 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:12.585505 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:47:12.585572 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0924 13:47:12.585915 (+ 343us) server_negotiation.cc:299] Negotiation successful
0924 13:47:12.585967 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":159,"thread_start_us":83,"threads_started":1}
I20260924 13:47:12.586862 14747 tablet_service.cc:1467] Processing CreateTablet for tablet 9521067550ef44ef8346f75b6f7519ef (DEFAULT_TABLE table=test-table [id=1d7b4d7079ef4b9aa3a17153b56efafe]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 13:47:12.586961 14611 tablet_service.cc:1467] Processing CreateTablet for tablet 9521067550ef44ef8346f75b6f7519ef (DEFAULT_TABLE table=test-table [id=1d7b4d7079ef4b9aa3a17153b56efafe]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 13:47:12.587257 14611 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9521067550ef44ef8346f75b6f7519ef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 13:47:12.587257 14747 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9521067550ef44ef8346f75b6f7519ef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 13:47:12.587347 14885 tablet_service.cc:1467] Processing CreateTablet for tablet 9521067550ef44ef8346f75b6f7519ef (DEFAULT_TABLE table=test-table [id=1d7b4d7079ef4b9aa3a17153b56efafe]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 13:47:12.587589 14885 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9521067550ef44ef8346f75b6f7519ef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 13:47:12.590551 14968 tablet_bootstrap.cc:492] T 9521067550ef44ef8346f75b6f7519ef P b417adcdc5bd4fc69acc675f3c57c03d: Bootstrap starting.
I20260924 13:47:12.590842 14969 tablet_bootstrap.cc:492] T 9521067550ef44ef8346f75b6f7519ef P 70e1bf096f5746f8adcc59a443d9e152: Bootstrap starting.
I20260924 13:47:12.591297 14968 tablet_bootstrap.cc:654] T 9521067550ef44ef8346f75b6f7519ef P b417adcdc5bd4fc69acc675f3c57c03d: Neither blocks nor log segments found. Creating new log.
I20260924 13:47:12.591419 14969 tablet_bootstrap.cc:654] T 9521067550ef44ef8346f75b6f7519ef P 70e1bf096f5746f8adcc59a443d9e152: Neither blocks nor log segments found. Creating new log.
I20260924 13:47:12.591805 14968 log.cc:824] T 9521067550ef44ef8346f75b6f7519ef P b417adcdc5bd4fc69acc675f3c57c03d: Log is configured to *not* fsync() on all Append() calls
I20260924 13:47:12.591892 14969 log.cc:824] T 9521067550ef44ef8346f75b6f7519ef P 70e1bf096f5746f8adcc59a443d9e152: Log is configured to *not* fsync() on all Append() calls
I20260924 13:47:12.592243 14970 tablet_bootstrap.cc:492] T 9521067550ef44ef8346f75b6f7519ef P 9b2cd463cecd4a5ba8144a03f54cbd34: Bootstrap starting.
I20260924 13:47:12.592592 14968 tablet_bootstrap.cc:492] T 9521067550ef44ef8346f75b6f7519ef P b417adcdc5bd4fc69acc675f3c57c03d: No bootstrap required, opened a new log
I20260924 13:47:12.592661 14968 ts_tablet_manager.cc:1397] T 9521067550ef44ef8346f75b6f7519ef P b417adcdc5bd4fc69acc675f3c57c03d: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260924 13:47:12.592831 14970 tablet_bootstrap.cc:654] T 9521067550ef44ef8346f75b6f7519ef P 9b2cd463cecd4a5ba8144a03f54cbd34: Neither blocks nor log segments found. Creating new log.
I20260924 13:47:12.593387 14970 log.cc:824] T 9521067550ef44ef8346f75b6f7519ef P 9b2cd463cecd4a5ba8144a03f54cbd34: Log is configured to *not* fsync() on all Append() calls
I20260924 13:47:12.593957 14969 tablet_bootstrap.cc:492] T 9521067550ef44ef8346f75b6f7519ef P 70e1bf096f5746f8adcc59a443d9e152: No bootstrap required, opened a new log
I20260924 13:47:12.594027 14969 ts_tablet_manager.cc:1397] T 9521067550ef44ef8346f75b6f7519ef P 70e1bf096f5746f8adcc59a443d9e152: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260924 13:47:12.594146 14970 tablet_bootstrap.cc:492] T 9521067550ef44ef8346f75b6f7519ef P 9b2cd463cecd4a5ba8144a03f54cbd34: No bootstrap required, opened a new log
I20260924 13:47:12.594225 14970 ts_tablet_manager.cc:1397] T 9521067550ef44ef8346f75b6f7519ef P 9b2cd463cecd4a5ba8144a03f54cbd34: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260924 13:47:12.594792 14968 raft_consensus.cc:348] T 9521067550ef44ef8346f75b6f7519ef P b417adcdc5bd4fc69acc675f3c57c03d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70e1bf096f5746f8adcc59a443d9e152" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 46527 } } peers { permanent_uuid: "9b2cd463cecd4a5ba8144a03f54cbd34" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 41825 } } peers { permanent_uuid: "b417adcdc5bd4fc69acc675f3c57c03d" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 45965 } }
I20260924 13:47:12.594988 14968 raft_consensus.cc:374] T 9521067550ef44ef8346f75b6f7519ef P b417adcdc5bd4fc69acc675f3c57c03d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 13:47:12.595026 14968 raft_consensus.cc:729] T 9521067550ef44ef8346f75b6f7519ef P b417adcdc5bd4fc69acc675f3c57c03d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b417adcdc5bd4fc69acc675f3c57c03d, State: Initialized, Role: FOLLOWER
I20260924 13:47:12.595180 14968 consensus_queue.cc:260] T 9521067550ef44ef8346f75b6f7519ef P b417adcdc5bd4fc69acc675f3c57c03d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70e1bf096f5746f8adcc59a443d9e152" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 46527 } } peers { permanent_uuid: "9b2cd463cecd4a5ba8144a03f54cbd34" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 41825 } } peers { permanent_uuid: "b417adcdc5bd4fc69acc675f3c57c03d" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 45965 } }
I20260924 13:47:12.595497 14951 heartbeater.cc:503] Master 127.13.149.254:42221 was elected leader, sending a full tablet report...
I20260924 13:47:12.595778 14968 ts_tablet_manager.cc:1428] T 9521067550ef44ef8346f75b6f7519ef P b417adcdc5bd4fc69acc675f3c57c03d: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260924 13:47:12.595701 14969 raft_consensus.cc:348] T 9521067550ef44ef8346f75b6f7519ef P 70e1bf096f5746f8adcc59a443d9e152 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70e1bf096f5746f8adcc59a443d9e152" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 46527 } } peers { permanent_uuid: "9b2cd463cecd4a5ba8144a03f54cbd34" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 41825 } } peers { permanent_uuid: "b417adcdc5bd4fc69acc675f3c57c03d" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 45965 } }
I20260924 13:47:12.595875 14969 raft_consensus.cc:374] T 9521067550ef44ef8346f75b6f7519ef P 70e1bf096f5746f8adcc59a443d9e152 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 13:47:12.595913 14969 raft_consensus.cc:729] T 9521067550ef44ef8346f75b6f7519ef P 70e1bf096f5746f8adcc59a443d9e152 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 70e1bf096f5746f8adcc59a443d9e152, State: Initialized, Role: FOLLOWER
I20260924 13:47:12.596047 14969 consensus_queue.cc:260] T 9521067550ef44ef8346f75b6f7519ef P 70e1bf096f5746f8adcc59a443d9e152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70e1bf096f5746f8adcc59a443d9e152" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 46527 } } peers { permanent_uuid: "9b2cd463cecd4a5ba8144a03f54cbd34" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 41825 } } peers { permanent_uuid: "b417adcdc5bd4fc69acc675f3c57c03d" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 45965 } }
I20260924 13:47:12.596225 14970 raft_consensus.cc:348] T 9521067550ef44ef8346f75b6f7519ef P 9b2cd463cecd4a5ba8144a03f54cbd34 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70e1bf096f5746f8adcc59a443d9e152" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 46527 } } peers { permanent_uuid: "9b2cd463cecd4a5ba8144a03f54cbd34" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 41825 } } peers { permanent_uuid: "b417adcdc5bd4fc69acc675f3c57c03d" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 45965 } }
I20260924 13:47:12.596362 14815 heartbeater.cc:503] Master 127.13.149.254:42221 was elected leader, sending a full tablet report...
I20260924 13:47:12.596400 14970 raft_consensus.cc:374] T 9521067550ef44ef8346f75b6f7519ef P 9b2cd463cecd4a5ba8144a03f54cbd34 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 13:47:12.596439 14970 raft_consensus.cc:729] T 9521067550ef44ef8346f75b6f7519ef P 9b2cd463cecd4a5ba8144a03f54cbd34 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9b2cd463cecd4a5ba8144a03f54cbd34, State: Initialized, Role: FOLLOWER
I20260924 13:47:12.596359 14969 ts_tablet_manager.cc:1428] T 9521067550ef44ef8346f75b6f7519ef P 70e1bf096f5746f8adcc59a443d9e152: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260924 13:47:12.596540 14970 consensus_queue.cc:260] T 9521067550ef44ef8346f75b6f7519ef P 9b2cd463cecd4a5ba8144a03f54cbd34 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70e1bf096f5746f8adcc59a443d9e152" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 46527 } } peers { permanent_uuid: "9b2cd463cecd4a5ba8144a03f54cbd34" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 41825 } } peers { permanent_uuid: "b417adcdc5bd4fc69acc675f3c57c03d" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 45965 } }
I20260924 13:47:12.596819 14677 heartbeater.cc:503] Master 127.13.149.254:42221 was elected leader, sending a full tablet report...
I20260924 13:47:12.596829 14970 ts_tablet_manager.cc:1428] T 9521067550ef44ef8346f75b6f7519ef P 9b2cd463cecd4a5ba8144a03f54cbd34: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
W20260924 13:47:12.625993 14816 tablet.cc:2369] T 9521067550ef44ef8346f75b6f7519ef P 70e1bf096f5746f8adcc59a443d9e152: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260924 13:47:12.640129 14976 raft_consensus.cc:482] T 9521067550ef44ef8346f75b6f7519ef P 9b2cd463cecd4a5ba8144a03f54cbd34 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 13:47:12.640280 14976 raft_consensus.cc:504] T 9521067550ef44ef8346f75b6f7519ef P 9b2cd463cecd4a5ba8144a03f54cbd34 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70e1bf096f5746f8adcc59a443d9e152" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 46527 } } peers { permanent_uuid: "9b2cd463cecd4a5ba8144a03f54cbd34" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 41825 } } peers { permanent_uuid: "b417adcdc5bd4fc69acc675f3c57c03d" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 45965 } }
I20260924 13:47:12.640642 14976 leader_election.cc:288] T 9521067550ef44ef8346f75b6f7519ef P 9b2cd463cecd4a5ba8144a03f54cbd34 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 70e1bf096f5746f8adcc59a443d9e152 (127.13.149.194:46527), b417adcdc5bd4fc69acc675f3c57c03d (127.13.149.195:45965)
I20260924 13:47:12.643460 14967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:12.640987 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.149.193:47897 (local address 127.13.149.195:45965)
0924 13:47:12.641025 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0924 13:47:12.641027 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:47:12.641035 (+ 8us) server_negotiation.cc:407] Connection header received
0924 13:47:12.641057 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:47:12.641058 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:47:12.641065 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:47:12.641109 (+ 44us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 13:47:12.641292 (+ 183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:12.641754 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:12.642709 (+ 955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:12.643120 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:12.643184 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:47:12.643193 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0924 13:47:12.643339 (+ 146us) server_negotiation.cc:299] Negotiation successful
0924 13:47:12.643374 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260924 13:47:12.643466 14964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:12.640896 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.149.193:45031 (local address 127.13.149.194:46527)
0924 13:47:12.640965 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0924 13:47:12.640967 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:47:12.641161 (+ 194us) server_negotiation.cc:407] Connection header received
0924 13:47:12.641224 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:47:12.641225 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:47:12.641234 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:47:12.641278 (+ 44us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 13:47:12.641410 (+ 132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:12.641871 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:12.642774 (+ 903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:12.643149 (+ 375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:12.643210 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:47:12.643218 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0924 13:47:12.643356 (+ 138us) server_negotiation.cc:299] Negotiation successful
0924 13:47:12.643391 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260924 13:47:12.643473 14679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:12.640818 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.195:45965 (local address 127.13.149.193:47897)
0924 13:47:12.640871 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0924 13:47:12.640882 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0924 13:47:12.640945 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:47:12.641131 (+ 186us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:47:12.641132 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:47:12.641134 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 13:47:12.641206 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:47:12.641208 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:47:12.641860 (+ 652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:47:12.641861 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:47:12.642616 (+ 755us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:47:12.642618 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:47:12.643201 (+ 583us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:47:12.643202 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:47:12.643276 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:47:12.643279 (+ 3us) client_negotiation.cc:769] Sending connection context
0924 13:47:12.643326 (+ 47us) client_negotiation.cc:240] Negotiation successful
0924 13:47:12.643372 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260924 13:47:12.643486 14977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:12.640819 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.194:46527 (local address 127.13.149.193:45031)
0924 13:47:12.641082 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0924 13:47:12.641091 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0924 13:47:12.641161 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:47:12.641287 (+ 126us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:47:12.641288 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:47:12.641289 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 13:47:12.641356 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:47:12.641357 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:47:12.641943 (+ 586us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:47:12.641944 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:47:12.642689 (+ 745us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:47:12.642691 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:47:12.643230 (+ 539us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:47:12.643231 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:47:12.643298 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:47:12.643301 (+ 3us) client_negotiation.cc:769] Sending connection context
0924 13:47:12.643346 (+ 45us) client_negotiation.cc:240] Negotiation successful
0924 13:47:12.643385 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":245,"mutex_wait_us":67,"thread_start_us":131,"threads_started":1}
I20260924 13:47:12.643815 14768 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9521067550ef44ef8346f75b6f7519ef" candidate_uuid: "9b2cd463cecd4a5ba8144a03f54cbd34" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "70e1bf096f5746f8adcc59a443d9e152" is_pre_election: true
I20260924 13:47:12.643846 14905 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9521067550ef44ef8346f75b6f7519ef" candidate_uuid: "9b2cd463cecd4a5ba8144a03f54cbd34" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b417adcdc5bd4fc69acc675f3c57c03d" is_pre_election: true
I20260924 13:47:12.643972 14768 raft_consensus.cc:2454] T 9521067550ef44ef8346f75b6f7519ef P 70e1bf096f5746f8adcc59a443d9e152 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9b2cd463cecd4a5ba8144a03f54cbd34 in term 0.
I20260924 13:47:12.643981 14905 raft_consensus.cc:2454] T 9521067550ef44ef8346f75b6f7519ef P b417adcdc5bd4fc69acc675f3c57c03d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9b2cd463cecd4a5ba8144a03f54cbd34 in term 0.
I20260924 13:47:12.644186 14567 leader_election.cc:302] T 9521067550ef44ef8346f75b6f7519ef P 9b2cd463cecd4a5ba8144a03f54cbd34 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 70e1bf096f5746f8adcc59a443d9e152, 9b2cd463cecd4a5ba8144a03f54cbd34; no voters:
I20260924 13:47:12.644335 14976 raft_consensus.cc:2781] T 9521067550ef44ef8346f75b6f7519ef P 9b2cd463cecd4a5ba8144a03f54cbd34 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 13:47:12.644402 14976 raft_consensus.cc:482] T 9521067550ef44ef8346f75b6f7519ef P 9b2cd463cecd4a5ba8144a03f54cbd34 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 13:47:12.644426 14976 raft_consensus.cc:3037] T 9521067550ef44ef8346f75b6f7519ef P 9b2cd463cecd4a5ba8144a03f54cbd34 [term 0 FOLLOWER]: Advancing to term 1
I20260924 13:47:12.645153 14976 raft_consensus.cc:504] T 9521067550ef44ef8346f75b6f7519ef P 9b2cd463cecd4a5ba8144a03f54cbd34 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70e1bf096f5746f8adcc59a443d9e152" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 46527 } } peers { permanent_uuid: "9b2cd463cecd4a5ba8144a03f54cbd34" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 41825 } } peers { permanent_uuid: "b417adcdc5bd4fc69acc675f3c57c03d" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 45965 } }
I20260924 13:47:12.645282 14976 leader_election.cc:288] T 9521067550ef44ef8346f75b6f7519ef P 9b2cd463cecd4a5ba8144a03f54cbd34 [CANDIDATE]: Term 1 election: Requested vote from peers 70e1bf096f5746f8adcc59a443d9e152 (127.13.149.194:46527), b417adcdc5bd4fc69acc675f3c57c03d (127.13.149.195:45965)
I20260924 13:47:12.645494 14768 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9521067550ef44ef8346f75b6f7519ef" candidate_uuid: "9b2cd463cecd4a5ba8144a03f54cbd34" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "70e1bf096f5746f8adcc59a443d9e152"
I20260924 13:47:12.645572 14768 raft_consensus.cc:3037] T 9521067550ef44ef8346f75b6f7519ef P 70e1bf096f5746f8adcc59a443d9e152 [term 0 FOLLOWER]: Advancing to term 1
I20260924 13:47:12.646349 14768 raft_consensus.cc:2454] T 9521067550ef44ef8346f75b6f7519ef P 70e1bf096f5746f8adcc59a443d9e152 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9b2cd463cecd4a5ba8144a03f54cbd34 in term 1.
I20260924 13:47:12.646456 14905 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9521067550ef44ef8346f75b6f7519ef" candidate_uuid: "9b2cd463cecd4a5ba8144a03f54cbd34" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b417adcdc5bd4fc69acc675f3c57c03d"
I20260924 13:47:12.646540 14567 leader_election.cc:302] T 9521067550ef44ef8346f75b6f7519ef P 9b2cd463cecd4a5ba8144a03f54cbd34 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 70e1bf096f5746f8adcc59a443d9e152, 9b2cd463cecd4a5ba8144a03f54cbd34; no voters:
I20260924 13:47:12.646543 14905 raft_consensus.cc:3037] T 9521067550ef44ef8346f75b6f7519ef P b417adcdc5bd4fc69acc675f3c57c03d [term 0 FOLLOWER]: Advancing to term 1
I20260924 13:47:12.646652 14976 raft_consensus.cc:2781] T 9521067550ef44ef8346f75b6f7519ef P 9b2cd463cecd4a5ba8144a03f54cbd34 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 13:47:12.646832 14976 raft_consensus.cc:686] T 9521067550ef44ef8346f75b6f7519ef P 9b2cd463cecd4a5ba8144a03f54cbd34 [term 1 LEADER]: Becoming Leader. State: Replica: 9b2cd463cecd4a5ba8144a03f54cbd34, State: Running, Role: LEADER
I20260924 13:47:12.646935 14976 consensus_queue.cc:237] T 9521067550ef44ef8346f75b6f7519ef P 9b2cd463cecd4a5ba8144a03f54cbd34 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70e1bf096f5746f8adcc59a443d9e152" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 46527 } } peers { permanent_uuid: "9b2cd463cecd4a5ba8144a03f54cbd34" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 41825 } } peers { permanent_uuid: "b417adcdc5bd4fc69acc675f3c57c03d" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 45965 } }
I20260924 13:47:12.647346 14905 raft_consensus.cc:2454] T 9521067550ef44ef8346f75b6f7519ef P b417adcdc5bd4fc69acc675f3c57c03d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9b2cd463cecd4a5ba8144a03f54cbd34 in term 1.
I20260924 13:47:12.647765 14485 catalog_manager.cc:5495] T 9521067550ef44ef8346f75b6f7519ef P 9b2cd463cecd4a5ba8144a03f54cbd34 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9b2cd463cecd4a5ba8144a03f54cbd34 (127.13.149.193). New cstate: current_term: 1 leader_uuid: "9b2cd463cecd4a5ba8144a03f54cbd34" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70e1bf096f5746f8adcc59a443d9e152" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 46527 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9b2cd463cecd4a5ba8144a03f54cbd34" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 41825 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b417adcdc5bd4fc69acc675f3c57c03d" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 45965 } health_report { overall_health: UNKNOWN } } }
Sep 24 13:47:12 dist-test-slave-sjl2 krb5kdc[14440](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257631, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.149.193@KRBTEST.COM
I20260924 13:47:12.667742 14966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:12.660430 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43406 (local address 127.13.149.193:41825)
0924 13:47:12.660517 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0924 13:47:12.660519 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:47:12.660579 (+ 60us) server_negotiation.cc:407] Connection header received
0924 13:47:12.660650 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:47:12.660651 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:47:12.660661 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:47:12.660708 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:47:12.661380 (+ 672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:12.661849 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:12.662535 (+ 686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:12.662734 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:12.662789 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:47:12.665073 (+ 2284us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:47:12.665089 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:47:12.665089 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 13:47:12.665097 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:47:12.666650 (+ 1553us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:47:12.667000 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:47:12.667004 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:47:12.667004 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:47:12.667059 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:47:12.667219 (+ 160us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:47:12.667220 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:47:12.667220 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:47:12.667371 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:47:12.667530 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0924 13:47:12.667566 (+ 36us) server_negotiation.cc:299] Negotiation successful
0924 13:47:12.667645 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
Sep 24 13:47:12 dist-test-slave-sjl2 krb5kdc[14440](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257632, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 24 13:47:12 dist-test-slave-sjl2 krb5kdc[14440](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257632, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.13.149.193@KRBTEST.COM
I20260924 13:47:12.689937 14966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:12.682746 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43420 (local address 127.13.149.193:41825)
0924 13:47:12.682825 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0924 13:47:12.682827 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:47:12.682863 (+ 36us) server_negotiation.cc:407] Connection header received
0924 13:47:12.682919 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:47:12.682920 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:47:12.682929 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:47:12.682978 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:47:12.683521 (+ 543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:12.684048 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:12.684758 (+ 710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:12.684986 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:12.685039 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:47:12.687273 (+ 2234us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:47:12.687274 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:47:12.687274 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 13:47:12.687281 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:47:12.688552 (+ 1271us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:47:12.689015 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:47:12.689016 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:47:12.689017 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:47:12.689071 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:47:12.689268 (+ 197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:47:12.689269 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:47:12.689269 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:47:12.689383 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:47:12.689470 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0924 13:47:12.689733 (+ 263us) server_negotiation.cc:299] Negotiation successful
0924 13:47:12.689832 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
Sep 24 13:47:12 dist-test-slave-sjl2 krb5kdc[14440](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257632, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 24 13:47:12 dist-test-slave-sjl2 krb5kdc[14440](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257632, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.149.193@KRBTEST.COM
I20260924 13:47:12.712376 14966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:12.705177 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43434 (local address 127.13.149.193:41825)
0924 13:47:12.705256 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0924 13:47:12.705258 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:47:12.705410 (+ 152us) server_negotiation.cc:407] Connection header received
0924 13:47:12.705440 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:47:12.705441 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:47:12.705452 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:47:12.705494 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:47:12.706112 (+ 618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:12.706594 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:12.707373 (+ 779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:12.707562 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:12.707620 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:47:12.710166 (+ 2546us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:47:12.710168 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:47:12.710168 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 13:47:12.710176 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:47:12.711440 (+ 1264us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:47:12.711798 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:47:12.711799 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:47:12.711799 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:47:12.711852 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:47:12.711980 (+ 128us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:47:12.711980 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:47:12.711980 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:47:12.712070 (+ 90us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:47:12.712145 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0924 13:47:12.712224 (+ 79us) server_negotiation.cc:299] Negotiation successful
0924 13:47:12.712308 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
W20260924 13:47:12.712848 14591 tablet_service.cc:2908] Rejecting scan request for tablet 9521067550ef44ef8346f75b6f7519ef: Uninitialized: clean time has not yet been initialized
I20260924 13:47:12.713698 13911 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJN9WRF/build/release/bin/kudu with pid 14548
I20260924 13:47:12.718654 13911 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJN9WRF/build/release/bin/kudu with pid 14682
I20260924 13:47:12.722921 13911 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJN9WRF/build/release/bin/kudu with pid 14820
I20260924 13:47:12.727293 13911 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJN9WRF/build/release/bin/kudu with pid 14453
2026-09-24T13:47:12Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (1895 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790257629041645-13911-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 13:47:12 dist-test-slave-sjl2 krb5kdc[14992](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 13:47:12 dist-test-slave-sjl2 krb5kdc[14992](info): set up 2 sockets
Sep 24 13:47:12 dist-test-slave-sjl2 krb5kdc[14992](info): commencing operation
krb5kdc: starting...
W20260924 13:47:13.693599 13911 mini_kdc.cc:121] Time spent starting KDC: real 0.946s user 0.003s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790257629041645-13911-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790257629041645-13911-0/krb5kdc/test-user.keytab.
Sep 24 13:47:13 dist-test-slave-sjl2 krb5kdc[14992](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257633, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T13:47:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T13:47:13Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.149.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.149.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.149.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.254.keytab.
Entry for principal kudu/127.13.149.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.254.keytab.
I20260924 13:47:13.812413 13911 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJN9WRF/build/release/bin/kudu
/tmp/dist-test-taskJN9WRF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790257629041645-13911-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790257629041645-13911-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790257629041645-13911-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790257629041645-13911-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.149.254:35189
--webserver_interface=127.13.149.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.254.keytab
--principal=kudu/127.13.149.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.149.212:40247
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.149.254:35189
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790257629041645-13911-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790257629041645-13911-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790257629041645-13911-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 13:47:13.893618 15005 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 13:47:13.893858 15005 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 13:47:13.893893 15005 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 13:47:13.895272 15005 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 13:47:13.895332 15005 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260924 13:47:13.895350 15005 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 13:47:13.895366 15005 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 13:47:13.895382 15005 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 13:47:13.895402 15005 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 13:47:13.897153 15005 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.149.212:40247
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790257629041645-13911-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790257629041645-13911-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.149.254:35189
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.254.keytab
--principal=kudu/127.13.149.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.149.254:35189
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790257629041645-13911-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.149.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15005
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790257629041645-13911-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 12:29:37 UTC on ccycloud.quasar-iqzbxn.root.comops.site
build id 4670106
I20260924 13:47:13.897435 15005 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 13:47:13.897665 15005 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 13:47:13.902952 15011 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 13:47:13.902967 15005 server_base.cc:1034] running on GCE node
W20260924 13:47:13.902904 15013 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 13:47:13.902904 15010 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 13:47:13.903431 15005 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 13:47:13.903761 15005 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 13:47:13.904968 15005 hybrid_clock.cc:648] HybridClock initialized: now 1790257633904948 us; error 39 us; skew 500 ppm
Sep 24 13:47:13 dist-test-slave-sjl2 krb5kdc[14992](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257633, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.149.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 13:47:13.907644 15005 init.cc:377] Logged in from keytab as kudu/127.13.149.254@KRBTEST.COM (short username kudu)
I20260924 13:47:13.908164 15005 webserver.cc:460] Webserver started at http://127.13.149.254:42115/ using document root <none> and password file <none>
I20260924 13:47:13.908409 15005 fs_manager.cc:359] Metadata directory not provided
I20260924 13:47:13.908463 15005 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 13:47:13.908591 15005 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 13:47:13.909754 15005 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790257629041645-13911-0/minicluster-data/master-0/data/instance:
uuid: "244154f1876f4f849fa9af203bd8fe2c"
format_stamp: "Formatted at 2026-09-24 13:47:13 on dist-test-slave-sjl2"
server_key: "9fb4d4e4d422694bb7d442042897ba0a"
server_key_iv: "5c1918b74e5f4d8e8a14ca14f300cd43"
server_key_version: "encryptionkey@0"
I20260924 13:47:13.910112 15005 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790257629041645-13911-0/minicluster-data/master-0/wal/instance:
uuid: "244154f1876f4f849fa9af203bd8fe2c"
format_stamp: "Formatted at 2026-09-24 13:47:13 on dist-test-slave-sjl2"
server_key: "9fb4d4e4d422694bb7d442042897ba0a"
server_key_iv: "5c1918b74e5f4d8e8a14ca14f300cd43"
server_key_version: "encryptionkey@0"
I20260924 13:47:13.911456 15005 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260924 13:47:13.912194 15020 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 13:47:13.912350 15005 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260924 13:47:13.912446 15005 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790257629041645-13911-0/minicluster-data/master-0/data,/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790257629041645-13911-0/minicluster-data/master-0/wal
uuid: "244154f1876f4f849fa9af203bd8fe2c"
format_stamp: "Formatted at 2026-09-24 13:47:13 on dist-test-slave-sjl2"
server_key: "9fb4d4e4d422694bb7d442042897ba0a"
server_key_iv: "5c1918b74e5f4d8e8a14ca14f300cd43"
server_key_version: "encryptionkey@0"
I20260924 13:47:13.912519 15005 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790257629041645-13911-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790257629041645-13911-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790257629041645-13911-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 13:47:13.925382 15005 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 13:47:13.927652 15005 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 13:47:13.927799 15005 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 13:47:13.932327 15005 rpc_server.cc:307] RPC server started. Bound to: 127.13.149.254:35189
I20260924 13:47:13.932390 15083 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.149.254:35189 every 8 connection(s)
I20260924 13:47:13.932719 15005 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790257629041645-13911-0/minicluster-data/master-0/data/info.pb
I20260924 13:47:13.933518 15084 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 13:47:13.935830 15084 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 244154f1876f4f849fa9af203bd8fe2c: Bootstrap starting.
I20260924 13:47:13.936399 15084 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 244154f1876f4f849fa9af203bd8fe2c: Neither blocks nor log segments found. Creating new log.
I20260924 13:47:13.936630 15084 log.cc:824] T 00000000000000000000000000000000 P 244154f1876f4f849fa9af203bd8fe2c: Log is configured to *not* fsync() on all Append() calls
I20260924 13:47:13.937311 13911 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJN9WRF/build/release/bin/kudu as pid 15005
I20260924 13:47:13.937407 13911 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790257629041645-13911-0/minicluster-data/master-0/wal/instance
I20260924 13:47:13.937443 15084 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 244154f1876f4f849fa9af203bd8fe2c: No bootstrap required, opened a new log
I20260924 13:47:13.937573 13911 external_mini_cluster.cc:1428] Setting key b59efecefe0843619dfe682e02bd9020
I20260924 13:47:13.940157 15084 raft_consensus.cc:348] T 00000000000000000000000000000000 P 244154f1876f4f849fa9af203bd8fe2c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "244154f1876f4f849fa9af203bd8fe2c" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 35189 } }
I20260924 13:47:13.940325 15084 raft_consensus.cc:374] T 00000000000000000000000000000000 P 244154f1876f4f849fa9af203bd8fe2c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 13:47:13.940361 15084 raft_consensus.cc:729] T 00000000000000000000000000000000 P 244154f1876f4f849fa9af203bd8fe2c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 244154f1876f4f849fa9af203bd8fe2c, State: Initialized, Role: FOLLOWER
I20260924 13:47:13.940490 15084 consensus_queue.cc:260] T 00000000000000000000000000000000 P 244154f1876f4f849fa9af203bd8fe2c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "244154f1876f4f849fa9af203bd8fe2c" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 35189 } }
I20260924 13:47:13.940565 15084 raft_consensus.cc:388] T 00000000000000000000000000000000 P 244154f1876f4f849fa9af203bd8fe2c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 13:47:13.940598 15084 raft_consensus.cc:482] T 00000000000000000000000000000000 P 244154f1876f4f849fa9af203bd8fe2c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 13:47:13.940646 15084 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 244154f1876f4f849fa9af203bd8fe2c [term 0 FOLLOWER]: Advancing to term 1
I20260924 13:47:13.941469 15084 raft_consensus.cc:504] T 00000000000000000000000000000000 P 244154f1876f4f849fa9af203bd8fe2c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "244154f1876f4f849fa9af203bd8fe2c" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 35189 } }
I20260924 13:47:13.941592 15084 leader_election.cc:302] T 00000000000000000000000000000000 P 244154f1876f4f849fa9af203bd8fe2c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 244154f1876f4f849fa9af203bd8fe2c; no voters:
I20260924 13:47:13.941776 15084 leader_election.cc:288] T 00000000000000000000000000000000 P 244154f1876f4f849fa9af203bd8fe2c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 13:47:13.941809 15089 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 244154f1876f4f849fa9af203bd8fe2c [term 1 FOLLOWER]: Leader election won for term 1
I20260924 13:47:13.941927 15089 raft_consensus.cc:686] T 00000000000000000000000000000000 P 244154f1876f4f849fa9af203bd8fe2c [term 1 LEADER]: Becoming Leader. State: Replica: 244154f1876f4f849fa9af203bd8fe2c, State: Running, Role: LEADER
I20260924 13:47:13.942015 15089 consensus_queue.cc:237] T 00000000000000000000000000000000 P 244154f1876f4f849fa9af203bd8fe2c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "244154f1876f4f849fa9af203bd8fe2c" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 35189 } }
I20260924 13:47:13.942070 15084 sys_catalog.cc:564] T 00000000000000000000000000000000 P 244154f1876f4f849fa9af203bd8fe2c [sys.catalog]: configured and running, proceeding with master startup.
I20260924 13:47:13.942348 15089 sys_catalog.cc:455] T 00000000000000000000000000000000 P 244154f1876f4f849fa9af203bd8fe2c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 244154f1876f4f849fa9af203bd8fe2c. Latest consensus state: current_term: 1 leader_uuid: "244154f1876f4f849fa9af203bd8fe2c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "244154f1876f4f849fa9af203bd8fe2c" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 35189 } } }
I20260924 13:47:13.942422 15089 sys_catalog.cc:458] T 00000000000000000000000000000000 P 244154f1876f4f849fa9af203bd8fe2c [sys.catalog]: This master's current role is: LEADER
I20260924 13:47:13.942363 15090 sys_catalog.cc:455] T 00000000000000000000000000000000 P 244154f1876f4f849fa9af203bd8fe2c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "244154f1876f4f849fa9af203bd8fe2c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "244154f1876f4f849fa9af203bd8fe2c" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 35189 } } }
I20260924 13:47:13.942618 15090 sys_catalog.cc:458] T 00000000000000000000000000000000 P 244154f1876f4f849fa9af203bd8fe2c [sys.catalog]: This master's current role is: LEADER
W20260924 13:47:13.943801 15105 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 244154f1876f4f849fa9af203bd8fe2c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260924 13:47:13.943859 15105 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260924 13:47:13.943902 15094 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 13:47:13.944057 15094 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Sep 24 13:47:13 dist-test-slave-sjl2 krb5kdc[14992](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257633, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.149.254@KRBTEST.COM
I20260924 13:47:13.945762 15094 catalog_manager.cc:1347] Generated new cluster ID: 0c49c33bba4348b3b96ae7589981c768
I20260924 13:47:13.945819 15094 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 13:47:13.949524 15088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:13.938085 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44706 (local address 127.13.149.254:35189)
0924 13:47:13.938244 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0924 13:47:13.938252 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0924 13:47:13.938278 (+ 26us) server_negotiation.cc:407] Connection header received
0924 13:47:13.938754 (+ 476us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:47:13.938761 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:47:13.938813 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:47:13.938944 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:47:13.939692 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:13.940721 (+ 1029us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:13.941726 (+ 1005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:13.942349 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:13.942817 (+ 468us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:47:13.945122 (+ 2305us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:47:13.945127 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:47:13.945128 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 13:47:13.945141 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:47:13.947220 (+ 2079us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:47:13.947648 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:47:13.947652 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:47:13.947653 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:47:13.947780 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:47:13.947961 (+ 181us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:47:13.947962 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:47:13.947963 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:47:13.948301 (+ 338us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:47:13.949026 (+ 725us) server_negotiation.cc:1035] Waiting for connection context
0924 13:47:13.949128 (+ 102us) server_negotiation.cc:299] Negotiation successful
0924 13:47:13.949294 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":107,"thread_start_us":65,"threads_started":1}
I20260924 13:47:13.960732 15094 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 13:47:13.961342 15094 catalog_manager.cc:1504] Loading token signing keys...
I20260924 13:47:13.963467 15094 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 244154f1876f4f849fa9af203bd8fe2c: Generated new TSK 0
I20260924 13:47:13.963652 15094 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.13.149.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.149.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.149.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.193.keytab.
Entry for principal kudu/127.13.149.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.193.keytab.
I20260924 13:47:13.992317 13911 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJN9WRF/build/release/bin/kudu
/tmp/dist-test-taskJN9WRF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790257629041645-13911-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790257629041645-13911-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790257629041645-13911-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790257629041645-13911-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.149.193:0
--local_ip_for_outbound_sockets=127.13.149.193
--webserver_interface=127.13.149.193
--webserver_port=0
--tserver_master_addrs=127.13.149.254:35189
--keytab_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.193.keytab
--principal=kudu/127.13.149.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.149.212:40247
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790257629041645-13911-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790257629041645-13911-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790257629041645-13911-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 13:47:14.076184 15110 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 13:47:14.076427 15110 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 13:47:14.076454 15110 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 13:47:14.077805 15110 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 13:47:14.077858 15110 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 13:47:14.077881 15110 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260924 13:47:14.077899 15110 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.149.193
I20260924 13:47:14.079423 15110 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.149.212:40247
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790257629041645-13911-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790257629041645-13911-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.193.keytab
--principal=kudu/127.13.149.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.149.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790257629041645-13911-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.149.193
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.13.149.254:35189
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15110
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.149.193
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790257629041645-13911-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 12:29:37 UTC on ccycloud.quasar-iqzbxn.root.comops.site
build id 4670106
I20260924 13:47:14.079635 15110 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 13:47:14.079836 15110 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 13:47:14.084815 15116 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 13:47:14.084789 15118 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 13:47:14.084791 15115 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 13:47:14.084842 15110 server_base.cc:1034] running on GCE node
I20260924 13:47:14.085238 15110 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 13:47:14.085521 15110 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 13:47:14.086673 15110 hybrid_clock.cc:648] HybridClock initialized: now 1790257634086650 us; error 36 us; skew 500 ppm
Sep 24 13:47:14 dist-test-slave-sjl2 krb5kdc[14992](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257634, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.149.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 13:47:14.089236 15110 init.cc:377] Logged in from keytab as kudu/127.13.149.193@KRBTEST.COM (short username kudu)
I20260924 13:47:14.089722 15110 webserver.cc:460] Webserver started at http://127.13.149.193:44361/ using document root <none> and password file <none>
I20260924 13:47:14.089929 15110 fs_manager.cc:359] Metadata directory not provided
I20260924 13:47:14.089977 15110 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 13:47:14.090087 15110 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 13:47:14.091164 15110 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790257629041645-13911-0/minicluster-data/ts-0/data/instance:
uuid: "21bfcfdb8dca41cca6a26000235952dd"
format_stamp: "Formatted at 2026-09-24 13:47:14 on dist-test-slave-sjl2"
server_key: "b2d81948074b4aaf2472db951f482bdd"
server_key_iv: "430ed41fa9c6807242473e901eba1286"
server_key_version: "encryptionkey@0"
I20260924 13:47:14.091512 15110 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790257629041645-13911-0/minicluster-data/ts-0/wal/instance:
uuid: "21bfcfdb8dca41cca6a26000235952dd"
format_stamp: "Formatted at 2026-09-24 13:47:14 on dist-test-slave-sjl2"
server_key: "b2d81948074b4aaf2472db951f482bdd"
server_key_iv: "430ed41fa9c6807242473e901eba1286"
server_key_version: "encryptionkey@0"
I20260924 13:47:14.092882 15110 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20260924 13:47:14.093709 15125 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 13:47:14.093871 15110 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 13:47:14.093962 15110 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790257629041645-13911-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790257629041645-13911-0/minicluster-data/ts-0/wal
uuid: "21bfcfdb8dca41cca6a26000235952dd"
format_stamp: "Formatted at 2026-09-24 13:47:14 on dist-test-slave-sjl2"
server_key: "b2d81948074b4aaf2472db951f482bdd"
server_key_iv: "430ed41fa9c6807242473e901eba1286"
server_key_version: "encryptionkey@0"
I20260924 13:47:14.094028 15110 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790257629041645-13911-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790257629041645-13911-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790257629041645-13911-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 13:47:14.160141 15110 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 13:47:14.162554 15110 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 13:47:14.162712 15110 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 13:47:14.163287 15110 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 13:47:14.163333 15110 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 13:47:14.163365 15110 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 13:47:14.163383 15110 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 13:47:14.170475 15110 rpc_server.cc:307] RPC server started. Bound to: 127.13.149.193:40107
I20260924 13:47:14.170548 15241 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.149.193:40107 every 8 connection(s)
I20260924 13:47:14.170881 15110 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790257629041645-13911-0/minicluster-data/ts-0/data/info.pb
Sep 24 13:47:14 dist-test-slave-sjl2 krb5kdc[14992](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257634, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.149.193@KRBTEST.COM for kudu/127.13.149.254@KRBTEST.COM
I20260924 13:47:14.174330 15088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:14.164129 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.149.193:55993 (local address 127.13.149.254:35189)
0924 13:47:14.164194 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0924 13:47:14.164197 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 13:47:14.164433 (+ 236us) server_negotiation.cc:407] Connection header received
0924 13:47:14.165150 (+ 717us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:47:14.165152 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:47:14.165166 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:47:14.165235 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:47:14.165950 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:14.166774 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:14.168470 (+ 1696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:14.168686 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:14.168753 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:47:14.171570 (+ 2817us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:47:14.171572 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:47:14.171572 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 13:47:14.171581 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:47:14.172853 (+ 1272us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:47:14.173310 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:47:14.173312 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:47:14.173312 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:47:14.173374 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:47:14.173614 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:47:14.173615 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:47:14.173616 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:47:14.173749 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:47:14.173809 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0924 13:47:14.174117 (+ 308us) server_negotiation.cc:299] Negotiation successful
0924 13:47:14.174231 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260924 13:47:14.174471 15134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:14.164110 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:35189 (local address 127.13.149.193:55993)
0924 13:47:14.164289 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0924 13:47:14.164333 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0924 13:47:14.164997 (+ 664us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:47:14.165324 (+ 327us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:47:14.165329 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:47:14.165641 (+ 312us) client_negotiation.cc:189] Negotiated authn=SASL
0924 13:47:14.165855 (+ 214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:47:14.165861 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:47:14.166918 (+ 1057us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:47:14.166920 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:47:14.168366 (+ 1446us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:47:14.168371 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:47:14.168785 (+ 414us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:47:14.168787 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:47:14.168979 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:47:14.169648 (+ 669us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 13:47:14.169659 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0924 13:47:14.171448 (+ 1789us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 13:47:14.172957 (+ 1509us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 13:47:14.172967 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 13:47:14.172970 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0924 13:47:14.173234 (+ 264us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 13:47:14.173430 (+ 196us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 13:47:14.173431 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 13:47:14.173432 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 13:47:14.173547 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 13:47:14.173809 (+ 262us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 13:47:14.173816 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 13:47:14.173982 (+ 166us) client_negotiation.cc:769] Sending connection context
0924 13:47:14.174071 (+ 89us) client_negotiation.cc:240] Negotiation successful
0924 13:47:14.174245 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":102,"threads_started":1}
I20260924 13:47:14.174975 15242 heartbeater.cc:347] Connected to a master server at 127.13.149.254:35189
I20260924 13:47:14.175057 15242 heartbeater.cc:464] Registering TS with master...
I20260924 13:47:14.175246 15242 heartbeater.cc:511] Master 127.13.149.254:35189 requested a full tablet report, sending...
I20260924 13:47:14.175778 15036 ts_manager.cc:194] Registered new tserver with Master: 21bfcfdb8dca41cca6a26000235952dd (127.13.149.193:40107)
I20260924 13:47:14.176693 15036 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.149.193@KRBTEST.COM'} at 127.13.149.193:55993
I20260924 13:47:14.178213 13911 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJN9WRF/build/release/bin/kudu as pid 15110
I20260924 13:47:14.178309 13911 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790257629041645-13911-0/minicluster-data/ts-0/wal/instance
I20260924 13:47:14.178503 13911 external_mini_cluster.cc:1428] Setting key 98f233622d6160850e58f1bf356201f7
I20260924 13:47:14.182116 15248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:14.176002 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:35189 (local address 127.13.149.193:54497)
0924 13:47:14.176141 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0924 13:47:14.176151 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 13:47:14.176207 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:47:14.176335 (+ 128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:47:14.176335 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:47:14.176542 (+ 207us) client_negotiation.cc:189] Negotiated authn=SASL
0924 13:47:14.176611 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:47:14.176613 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:47:14.177399 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:47:14.177400 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:47:14.177955 (+ 555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:47:14.177957 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:47:14.178306 (+ 349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:47:14.178307 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:47:14.178411 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:47:14.178906 (+ 495us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 13:47:14.178911 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 13:47:14.179307 (+ 396us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 13:47:14.181102 (+ 1795us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 13:47:14.181103 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 13:47:14.181104 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 13:47:14.181338 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 13:47:14.181505 (+ 167us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 13:47:14.181506 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 13:47:14.181506 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 13:47:14.181570 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 13:47:14.181815 (+ 245us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 13:47:14.181816 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 13:47:14.181865 (+ 49us) client_negotiation.cc:769] Sending connection context
0924 13:47:14.181940 (+ 75us) client_negotiation.cc:240] Negotiation successful
0924 13:47:14.182034 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":70,"threads_started":1}
I20260924 13:47:14.182153 15088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:14.176061 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.149.193:54497 (local address 127.13.149.254:35189)
0924 13:47:14.176100 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0924 13:47:14.176101 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0924 13:47:14.176193 (+ 92us) server_negotiation.cc:407] Connection header received
0924 13:47:14.176273 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:47:14.176274 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:47:14.176281 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:47:14.176329 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:47:14.176671 (+ 342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:14.177315 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:14.178045 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:14.178233 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:14.178293 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:47:14.179384 (+ 1091us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:47:14.179385 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:47:14.179385 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 13:47:14.179392 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:47:14.181002 (+ 1610us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:47:14.181409 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:47:14.181409 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:47:14.181410 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:47:14.181456 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:47:14.181620 (+ 164us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:47:14.181620 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:47:14.181620 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:47:14.181715 (+ 95us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:47:14.181758 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0924 13:47:14.181981 (+ 223us) server_negotiation.cc:299] Negotiation successful
0924 13:47:14.182062 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
WARNING: no policy specified for kudu/127.13.149.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.149.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.149.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.194.keytab.
Entry for principal kudu/127.13.149.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.194.keytab.
I20260924 13:47:14.203909 13911 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJN9WRF/build/release/bin/kudu
/tmp/dist-test-taskJN9WRF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790257629041645-13911-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790257629041645-13911-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790257629041645-13911-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790257629041645-13911-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.149.194:0
--local_ip_for_outbound_sockets=127.13.149.194
--webserver_interface=127.13.149.194
--webserver_port=0
--tserver_master_addrs=127.13.149.254:35189
--keytab_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.194.keytab
--principal=kudu/127.13.149.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.149.212:40247
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790257629041645-13911-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790257629041645-13911-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790257629041645-13911-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 13:47:14.286538 15251 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 13:47:14.286782 15251 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 13:47:14.286811 15251 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 13:47:14.288087 15251 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 13:47:14.288141 15251 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 13:47:14.288163 15251 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260924 13:47:14.288180 15251 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.149.194
I20260924 13:47:14.289791 15251 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.149.212:40247
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790257629041645-13911-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790257629041645-13911-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.194.keytab
--principal=kudu/127.13.149.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.149.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790257629041645-13911-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.149.194
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.13.149.254:35189
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15251
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.149.194
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790257629041645-13911-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 12:29:37 UTC on ccycloud.quasar-iqzbxn.root.comops.site
build id 4670106
I20260924 13:47:14.290050 15251 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 13:47:14.290287 15251 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 13:47:14.295141 15256 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 13:47:14.295185 15257 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 13:47:14.295228 15259 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 13:47:14.295394 15251 server_base.cc:1034] running on GCE node
I20260924 13:47:14.295701 15251 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 13:47:14.295954 15251 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 13:47:14.297092 15251 hybrid_clock.cc:648] HybridClock initialized: now 1790257634297078 us; error 33 us; skew 500 ppm
Sep 24 13:47:14 dist-test-slave-sjl2 krb5kdc[14992](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257634, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.149.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 13:47:14.299688 15251 init.cc:377] Logged in from keytab as kudu/127.13.149.194@KRBTEST.COM (short username kudu)
I20260924 13:47:14.300112 15251 webserver.cc:460] Webserver started at http://127.13.149.194:37917/ using document root <none> and password file <none>
I20260924 13:47:14.300328 15251 fs_manager.cc:359] Metadata directory not provided
I20260924 13:47:14.300369 15251 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 13:47:14.300464 15251 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 13:47:14.301533 15251 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790257629041645-13911-0/minicluster-data/ts-1/data/instance:
uuid: "2d405e4bdb8141389defffbc73190a5b"
format_stamp: "Formatted at 2026-09-24 13:47:14 on dist-test-slave-sjl2"
server_key: "18ed439deac443db30e05cd365c01790"
server_key_iv: "f7f947d1e913ef653025f01dbc24f301"
server_key_version: "encryptionkey@0"
I20260924 13:47:14.301838 15251 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790257629041645-13911-0/minicluster-data/ts-1/wal/instance:
uuid: "2d405e4bdb8141389defffbc73190a5b"
format_stamp: "Formatted at 2026-09-24 13:47:14 on dist-test-slave-sjl2"
server_key: "18ed439deac443db30e05cd365c01790"
server_key_iv: "f7f947d1e913ef653025f01dbc24f301"
server_key_version: "encryptionkey@0"
I20260924 13:47:14.303102 15251 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260924 13:47:14.303815 15266 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 13:47:14.303983 15251 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 13:47:14.304068 15251 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790257629041645-13911-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790257629041645-13911-0/minicluster-data/ts-1/wal
uuid: "2d405e4bdb8141389defffbc73190a5b"
format_stamp: "Formatted at 2026-09-24 13:47:14 on dist-test-slave-sjl2"
server_key: "18ed439deac443db30e05cd365c01790"
server_key_iv: "f7f947d1e913ef653025f01dbc24f301"
server_key_version: "encryptionkey@0"
I20260924 13:47:14.304129 15251 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790257629041645-13911-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790257629041645-13911-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790257629041645-13911-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 13:47:14.324468 15251 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 13:47:14.326617 15251 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 13:47:14.326742 15251 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 13:47:14.327205 15251 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 13:47:14.327244 15251 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 13:47:14.327268 15251 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 13:47:14.327280 15251 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 13:47:14.333877 15251 rpc_server.cc:307] RPC server started. Bound to: 127.13.149.194:36491
I20260924 13:47:14.333957 15386 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.149.194:36491 every 8 connection(s)
I20260924 13:47:14.334240 15251 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790257629041645-13911-0/minicluster-data/ts-1/data/info.pb
Sep 24 13:47:14 dist-test-slave-sjl2 krb5kdc[14992](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257634, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.149.194@KRBTEST.COM for kudu/127.13.149.254@KRBTEST.COM
I20260924 13:47:14.337065 15088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:14.327723 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.149.194:40511 (local address 127.13.149.254:35189)
0924 13:47:14.327774 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0924 13:47:14.327776 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:47:14.327997 (+ 221us) server_negotiation.cc:407] Connection header received
0924 13:47:14.328504 (+ 507us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:47:14.328505 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:47:14.328518 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:47:14.328589 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:47:14.329582 (+ 993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:14.330371 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:14.331918 (+ 1547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:14.332189 (+ 271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:14.332245 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:47:14.334912 (+ 2667us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:47:14.334914 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:47:14.334914 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 13:47:14.334921 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:47:14.335968 (+ 1047us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:47:14.336323 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:47:14.336324 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:47:14.336325 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:47:14.336368 (+ 43us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:47:14.336527 (+ 159us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:47:14.336527 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:47:14.336528 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:47:14.336627 (+ 99us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:47:14.336671 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0924 13:47:14.336902 (+ 231us) server_negotiation.cc:299] Negotiation successful
0924 13:47:14.336999 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260924 13:47:14.337283 15275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:14.327728 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:35189 (local address 127.13.149.194:40511)
0924 13:47:14.327879 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0924 13:47:14.327916 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0924 13:47:14.328409 (+ 493us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:47:14.328733 (+ 324us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:47:14.328738 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:47:14.329138 (+ 400us) client_negotiation.cc:189] Negotiated authn=SASL
0924 13:47:14.329464 (+ 326us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:47:14.329475 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:47:14.330486 (+ 1011us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:47:14.330488 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:47:14.331828 (+ 1340us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:47:14.331831 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:47:14.332314 (+ 483us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:47:14.332315 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:47:14.332470 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:47:14.333141 (+ 671us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 13:47:14.333152 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0924 13:47:14.334819 (+ 1667us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 13:47:14.336045 (+ 1226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 13:47:14.336051 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 13:47:14.336053 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 13:47:14.336269 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 13:47:14.336405 (+ 136us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 13:47:14.336405 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 13:47:14.336405 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 13:47:14.336489 (+ 84us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 13:47:14.336669 (+ 180us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 13:47:14.336672 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 13:47:14.336802 (+ 130us) client_negotiation.cc:769] Sending connection context
0924 13:47:14.336882 (+ 80us) client_negotiation.cc:240] Negotiation successful
0924 13:47:14.337060 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":89,"threads_started":1}
I20260924 13:47:14.337724 15387 heartbeater.cc:347] Connected to a master server at 127.13.149.254:35189
I20260924 13:47:14.337809 15387 heartbeater.cc:464] Registering TS with master...
I20260924 13:47:14.337980 15387 heartbeater.cc:511] Master 127.13.149.254:35189 requested a full tablet report, sending...
I20260924 13:47:14.338354 15036 ts_manager.cc:194] Registered new tserver with Master: 2d405e4bdb8141389defffbc73190a5b (127.13.149.194:36491)
I20260924 13:47:14.338759 15036 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.149.194@KRBTEST.COM'} at 127.13.149.194:40511
I20260924 13:47:14.338997 13911 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJN9WRF/build/release/bin/kudu as pid 15251
I20260924 13:47:14.339090 13911 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790257629041645-13911-0/minicluster-data/ts-1/wal/instance
I20260924 13:47:14.339251 13911 external_mini_cluster.cc:1428] Setting key 32c769b7c0ee69f11aca76f94fea3dba
I20260924 13:47:14.344854 15393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:14.338682 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:35189 (local address 127.13.149.194:58961)
0924 13:47:14.338799 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0924 13:47:14.338810 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0924 13:47:14.338863 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:47:14.338962 (+ 99us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:47:14.338962 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:47:14.339156 (+ 194us) client_negotiation.cc:189] Negotiated authn=SASL
0924 13:47:14.339232 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:47:14.339234 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:47:14.340009 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:47:14.340010 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:47:14.340543 (+ 533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:47:14.340544 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:47:14.340988 (+ 444us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:47:14.340989 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:47:14.341063 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:47:14.341472 (+ 409us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 13:47:14.341478 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0924 13:47:14.341873 (+ 395us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 13:47:14.343788 (+ 1915us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 13:47:14.343789 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 13:47:14.343789 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 13:47:14.343961 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 13:47:14.344162 (+ 201us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 13:47:14.344163 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 13:47:14.344163 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 13:47:14.344216 (+ 53us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 13:47:14.344530 (+ 314us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 13:47:14.344531 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 13:47:14.344584 (+ 53us) client_negotiation.cc:769] Sending connection context
0924 13:47:14.344631 (+ 47us) client_negotiation.cc:240] Negotiation successful
0924 13:47:14.344776 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":89,"thread_start_us":60,"threads_started":1}
I20260924 13:47:14.344853 15088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:14.338756 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.149.194:58961 (local address 127.13.149.254:35189)
0924 13:47:14.338879 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0924 13:47:14.338881 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:47:14.338888 (+ 7us) server_negotiation.cc:407] Connection header received
0924 13:47:14.338905 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:47:14.338906 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:47:14.338915 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:47:14.338958 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:47:14.339318 (+ 360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:14.339939 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:14.340622 (+ 683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:14.340882 (+ 260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:14.340985 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:47:14.341988 (+ 1003us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:47:14.341990 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:47:14.341990 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 13:47:14.341998 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:47:14.343696 (+ 1698us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:47:14.344031 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:47:14.344033 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:47:14.344033 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:47:14.344093 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:47:14.344270 (+ 177us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:47:14.344271 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:47:14.344271 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:47:14.344472 (+ 201us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:47:14.344544 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0924 13:47:14.344674 (+ 130us) server_negotiation.cc:299] Negotiation successful
0924 13:47:14.344755 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":106}
WARNING: no policy specified for kudu/127.13.149.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.149.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.149.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.195.keytab.
Entry for principal kudu/127.13.149.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.195.keytab.
I20260924 13:47:14.363626 13911 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJN9WRF/build/release/bin/kudu
/tmp/dist-test-taskJN9WRF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790257629041645-13911-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790257629041645-13911-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790257629041645-13911-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790257629041645-13911-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.149.195:0
--local_ip_for_outbound_sockets=127.13.149.195
--webserver_interface=127.13.149.195
--webserver_port=0
--tserver_master_addrs=127.13.149.254:35189
--keytab_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.195.keytab
--principal=kudu/127.13.149.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.149.212:40247
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790257629041645-13911-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790257629041645-13911-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790257629041645-13911-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 13:47:14.445564 15396 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 13:47:14.445811 15396 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 13:47:14.445847 15396 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 13:47:14.447124 15396 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 13:47:14.447176 15396 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 13:47:14.447206 15396 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260924 13:47:14.447232 15396 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.149.195
I20260924 13:47:14.448877 15396 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.149.212:40247
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790257629041645-13911-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790257629041645-13911-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.195.keytab
--principal=kudu/127.13.149.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.149.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790257629041645-13911-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.149.195
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.13.149.254:35189
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15396
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.149.195
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790257629041645-13911-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 12:29:37 UTC on ccycloud.quasar-iqzbxn.root.comops.site
build id 4670106
I20260924 13:47:14.449196 15396 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 13:47:14.449426 15396 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 13:47:14.454718 15401 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 13:47:14.454766 15402 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 13:47:14.454952 15396 server_base.cc:1034] running on GCE node
W20260924 13:47:14.454770 15404 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 13:47:14.455204 15396 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 13:47:14.455432 15396 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 13:47:14.456604 15396 hybrid_clock.cc:648] HybridClock initialized: now 1790257634456586 us; error 30 us; skew 500 ppm
Sep 24 13:47:14 dist-test-slave-sjl2 krb5kdc[14992](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257634, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.149.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 13:47:14.459504 15396 init.cc:377] Logged in from keytab as kudu/127.13.149.195@KRBTEST.COM (short username kudu)
I20260924 13:47:14.460043 15396 webserver.cc:460] Webserver started at http://127.13.149.195:33687/ using document root <none> and password file <none>
I20260924 13:47:14.460300 15396 fs_manager.cc:359] Metadata directory not provided
I20260924 13:47:14.460359 15396 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 13:47:14.460500 15396 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 13:47:14.461701 15396 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790257629041645-13911-0/minicluster-data/ts-2/data/instance:
uuid: "2ab1deb934114642a088159e52c51b56"
format_stamp: "Formatted at 2026-09-24 13:47:14 on dist-test-slave-sjl2"
server_key: "8558253e8ad4bdfa5dbf73b08dd47020"
server_key_iv: "f7fa5d8a5be029af5de4af825d27bc67"
server_key_version: "encryptionkey@0"
I20260924 13:47:14.462100 15396 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790257629041645-13911-0/minicluster-data/ts-2/wal/instance:
uuid: "2ab1deb934114642a088159e52c51b56"
format_stamp: "Formatted at 2026-09-24 13:47:14 on dist-test-slave-sjl2"
server_key: "8558253e8ad4bdfa5dbf73b08dd47020"
server_key_iv: "f7fa5d8a5be029af5de4af825d27bc67"
server_key_version: "encryptionkey@0"
I20260924 13:47:14.463634 15396 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260924 13:47:14.464452 15411 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 13:47:14.464675 15396 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260924 13:47:14.464769 15396 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790257629041645-13911-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790257629041645-13911-0/minicluster-data/ts-2/wal
uuid: "2ab1deb934114642a088159e52c51b56"
format_stamp: "Formatted at 2026-09-24 13:47:14 on dist-test-slave-sjl2"
server_key: "8558253e8ad4bdfa5dbf73b08dd47020"
server_key_iv: "f7fa5d8a5be029af5de4af825d27bc67"
server_key_version: "encryptionkey@0"
I20260924 13:47:14.464864 15396 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790257629041645-13911-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790257629041645-13911-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790257629041645-13911-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 13:47:14.479372 15396 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 13:47:14.481427 15396 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 13:47:14.481550 15396 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 13:47:14.482107 15396 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 13:47:14.482151 15396 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 13:47:14.482182 15396 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 13:47:14.482197 15396 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 13:47:14.489791 15396 rpc_server.cc:307] RPC server started. Bound to: 127.13.149.195:37565
I20260924 13:47:14.489944 15527 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.149.195:37565 every 8 connection(s)
Sep 24 13:47:14 dist-test-slave-sjl2 krb5kdc[14992](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257634, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.149.195@KRBTEST.COM for kudu/127.13.149.254@KRBTEST.COM
I20260924 13:47:14.490219 15396 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790257629041645-13911-0/minicluster-data/ts-2/data/info.pb
I20260924 13:47:14.493491 15088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:14.482553 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.149.195:59435 (local address 127.13.149.254:35189)
0924 13:47:14.482618 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0924 13:47:14.482621 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 13:47:14.482835 (+ 214us) server_negotiation.cc:407] Connection header received
0924 13:47:14.483561 (+ 726us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:47:14.483563 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:47:14.483577 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:47:14.483668 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:47:14.484630 (+ 962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:14.485634 (+ 1004us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:14.487369 (+ 1735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:14.487696 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:14.487791 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:47:14.490716 (+ 2925us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:47:14.490718 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:47:14.490718 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 13:47:14.490727 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:47:14.492106 (+ 1379us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:47:14.492519 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:47:14.492521 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:47:14.492522 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:47:14.492576 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:47:14.492814 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:47:14.492815 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:47:14.492815 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:47:14.492978 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:47:14.493039 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0924 13:47:14.493311 (+ 272us) server_negotiation.cc:299] Negotiation successful
0924 13:47:14.493398 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260924 13:47:14.493682 15420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:14.482528 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:35189 (local address 127.13.149.195:59435)
0924 13:47:14.482696 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0924 13:47:14.482738 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0924 13:47:14.483425 (+ 687us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:47:14.483815 (+ 390us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:47:14.483821 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:47:14.484207 (+ 386us) client_negotiation.cc:189] Negotiated authn=SASL
0924 13:47:14.484537 (+ 330us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:47:14.484545 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:47:14.485856 (+ 1311us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:47:14.485857 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:47:14.487238 (+ 1381us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:47:14.487244 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:47:14.487802 (+ 558us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:47:14.487804 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:47:14.487963 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:47:14.488730 (+ 767us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 13:47:14.488743 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0924 13:47:14.490586 (+ 1843us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 13:47:14.492193 (+ 1607us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 13:47:14.492202 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 13:47:14.492205 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0924 13:47:14.492449 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 13:47:14.492630 (+ 181us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 13:47:14.492631 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 13:47:14.492631 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 13:47:14.492756 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 13:47:14.493029 (+ 273us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 13:47:14.493035 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 13:47:14.493185 (+ 150us) client_negotiation.cc:769] Sending connection context
0924 13:47:14.493280 (+ 95us) client_negotiation.cc:240] Negotiation successful
0924 13:47:14.493455 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":77,"threads_started":1}
I20260924 13:47:14.494132 15528 heartbeater.cc:347] Connected to a master server at 127.13.149.254:35189
I20260924 13:47:14.494217 15528 heartbeater.cc:464] Registering TS with master...
I20260924 13:47:14.494405 15528 heartbeater.cc:511] Master 127.13.149.254:35189 requested a full tablet report, sending...
I20260924 13:47:14.494836 15037 ts_manager.cc:194] Registered new tserver with Master: 2ab1deb934114642a088159e52c51b56 (127.13.149.195:37565)
I20260924 13:47:14.495344 15037 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.149.195@KRBTEST.COM'} at 127.13.149.195:59435
I20260924 13:47:14.498812 13911 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJN9WRF/build/release/bin/kudu as pid 15396
I20260924 13:47:14.498901 13911 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790257629041645-13911-0/minicluster-data/ts-2/wal/instance
I20260924 13:47:14.499043 13911 external_mini_cluster.cc:1428] Setting key af720f14a0fe97d07795599aa7fe5a0a
I20260924 13:47:14.500017 13911 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260924 13:47:14.501451 15088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:14.495228 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.149.195:60723 (local address 127.13.149.254:35189)
0924 13:47:14.495272 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0924 13:47:14.495274 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:47:14.495339 (+ 65us) server_negotiation.cc:407] Connection header received
0924 13:47:14.495385 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:47:14.495386 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:47:14.495396 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:47:14.495456 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:47:14.495905 (+ 449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:14.496522 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:14.497284 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:14.497522 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:14.497579 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:47:14.498788 (+ 1209us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:47:14.498790 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:47:14.498790 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 13:47:14.498797 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:47:14.500351 (+ 1554us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:47:14.500734 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:47:14.500735 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:47:14.500735 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:47:14.500786 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:47:14.501006 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:47:14.501007 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:47:14.501007 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:47:14.501120 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:47:14.501167 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0924 13:47:14.501321 (+ 154us) server_negotiation.cc:299] Negotiation successful
0924 13:47:14.501385 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260924 13:47:14.501518 15534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:14.495161 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:35189 (local address 127.13.149.195:60723)
0924 13:47:14.495284 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0924 13:47:14.495295 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0924 13:47:14.495347 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:47:14.495502 (+ 155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:47:14.495503 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:47:14.495721 (+ 218us) client_negotiation.cc:189] Negotiated authn=SASL
0924 13:47:14.495797 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:47:14.495800 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:47:14.496620 (+ 820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:47:14.496621 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:47:14.497210 (+ 589us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:47:14.497212 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:47:14.497581 (+ 369us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:47:14.497582 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:47:14.497658 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:47:14.498293 (+ 635us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 13:47:14.498298 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 13:47:14.498659 (+ 361us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 13:47:14.500438 (+ 1779us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 13:47:14.500439 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 13:47:14.500439 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 13:47:14.500663 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 13:47:14.500856 (+ 193us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 13:47:14.500857 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 13:47:14.500857 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 13:47:14.500950 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 13:47:14.501168 (+ 218us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 13:47:14.501169 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 13:47:14.501224 (+ 55us) client_negotiation.cc:769] Sending connection context
0924 13:47:14.501301 (+ 77us) client_negotiation.cc:240] Negotiation successful
0924 13:47:14.501412 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":68,"threads_started":1}
Sep 24 13:47:14 dist-test-slave-sjl2 krb5kdc[14992](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257634, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 24 13:47:14 dist-test-slave-sjl2 krb5kdc[14992](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257634, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.13.149.254@KRBTEST.COM
I20260924 13:47:14.524447 15088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:14.516149 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44722 (local address 127.13.149.254:35189)
0924 13:47:14.516251 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0924 13:47:14.516254 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 13:47:14.516303 (+ 49us) server_negotiation.cc:407] Connection header received
0924 13:47:14.516372 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:47:14.516373 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:47:14.516383 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:47:14.516427 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:47:14.517100 (+ 673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:14.517609 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:14.518564 (+ 955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:14.518756 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:14.518824 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:47:14.521783 (+ 2959us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:47:14.521784 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:47:14.521785 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 13:47:14.521793 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:47:14.523140 (+ 1347us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:47:14.523539 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:47:14.523540 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:47:14.523540 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:47:14.523587 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:47:14.523861 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:47:14.523863 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:47:14.523863 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:47:14.523997 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:47:14.524087 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0924 13:47:14.524282 (+ 195us) server_negotiation.cc:299] Negotiation successful
0924 13:47:14.524374 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260924 13:47:14.526329 15037 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44722:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 13:47:14.528003 15037 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 13:47:14.535755 15546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:14.531169 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.193:40107 (local address 127.0.0.1:38352)
0924 13:47:14.531352 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0924 13:47:14.531363 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0924 13:47:14.531417 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:47:14.532163 (+ 746us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:47:14.532165 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:47:14.532167 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 13:47:14.532225 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:47:14.532227 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:47:14.533106 (+ 879us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:47:14.533107 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:47:14.534467 (+ 1360us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:47:14.534470 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:47:14.535408 (+ 938us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:47:14.535409 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:47:14.535496 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:47:14.535512 (+ 16us) client_negotiation.cc:769] Sending connection context
0924 13:47:14.535581 (+ 69us) client_negotiation.cc:240] Negotiation successful
0924 13:47:14.535645 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":101,"threads_started":1}
I20260924 13:47:14.535755 15550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:14.531668 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38352 (local address 127.13.149.193:40107)
0924 13:47:14.531839 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0924 13:47:14.531844 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 13:47:14.531857 (+ 13us) server_negotiation.cc:407] Connection header received
0924 13:47:14.531906 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:47:14.531908 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:47:14.532065 (+ 157us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:47:14.532156 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 13:47:14.532287 (+ 131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:14.533035 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:14.534573 (+ 1538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:14.535330 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:14.535399 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:47:14.535465 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0924 13:47:14.535602 (+ 137us) server_negotiation.cc:299] Negotiation successful
0924 13:47:14.535645 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":141,"thread_start_us":67,"threads_started":1}
I20260924 13:47:14.536114 15548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:14.531364 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.195:37565 (local address 127.0.0.1:47654)
0924 13:47:14.531603 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0924 13:47:14.531614 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0924 13:47:14.531666 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:47:14.532297 (+ 631us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:47:14.532297 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:47:14.532298 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 13:47:14.532319 (+ 21us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:47:14.532320 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:47:14.533628 (+ 1308us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:47:14.533629 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:47:14.534872 (+ 1243us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:47:14.534874 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:47:14.535823 (+ 949us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:47:14.535825 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:47:14.535888 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:47:14.535892 (+ 4us) client_negotiation.cc:769] Sending connection context
0924 13:47:14.535920 (+ 28us) client_negotiation.cc:240] Negotiation successful
0924 13:47:14.535969 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":210,"mutex_wait_us":35,"thread_start_us":54,"threads_started":1}
I20260924 13:47:14.536258 15547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:14.531271 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41768 (local address 127.13.149.194:36491)
0924 13:47:14.531797 (+ 526us) server_negotiation.cc:206] Beginning negotiation
0924 13:47:14.531803 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0924 13:47:14.531816 (+ 13us) server_negotiation.cc:407] Connection header received
0924 13:47:14.531862 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:47:14.531865 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:47:14.531883 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:47:14.531977 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 13:47:14.532387 (+ 410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:14.533553 (+ 1166us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:14.534990 (+ 1437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:14.535725 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:14.535795 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:47:14.535851 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0924 13:47:14.536106 (+ 255us) server_negotiation.cc:299] Negotiation successful
0924 13:47:14.536149 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":480,"thread_start_us":72,"threads_started":1}
I20260924 13:47:14.536298 15549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:14.531612 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47654 (local address 127.13.149.195:37565)
0924 13:47:14.531932 (+ 320us) server_negotiation.cc:206] Beginning negotiation
0924 13:47:14.531936 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 13:47:14.531945 (+ 9us) server_negotiation.cc:407] Connection header received
0924 13:47:14.531979 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:47:14.531981 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:47:14.531994 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:47:14.532145 (+ 151us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 13:47:14.532372 (+ 227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:14.533533 (+ 1161us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:14.534949 (+ 1416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:14.535708 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:14.536066 (+ 358us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:47:14.536138 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0924 13:47:14.536161 (+ 23us) server_negotiation.cc:299] Negotiation successful
0924 13:47:14.536205 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":288,"thread_start_us":137,"threads_started":1}
I20260924 13:47:14.536413 15545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:14.530982 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.194:36491 (local address 127.0.0.1:41768)
0924 13:47:14.531187 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0924 13:47:14.531262 (+ 75us) client_negotiation.cc:174] Beginning negotiation
0924 13:47:14.531399 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:47:14.531981 (+ 582us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:47:14.531985 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:47:14.531988 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 13:47:14.532080 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:47:14.532082 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:47:14.533642 (+ 1560us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:47:14.533643 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:47:14.534908 (+ 1265us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:47:14.534912 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:47:14.535879 (+ 967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:47:14.535879 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:47:14.535943 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:47:14.535946 (+ 3us) client_negotiation.cc:769] Sending connection context
0924 13:47:14.535983 (+ 37us) client_negotiation.cc:240] Negotiation successful
0924 13:47:14.536017 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"mutex_wait_us":132,"thread_start_us":64,"threads_started":1}
I20260924 13:47:14.536969 15462 tablet_service.cc:1467] Processing CreateTablet for tablet fe2e7ae7153c4fe7a46830ffc8248a52 (DEFAULT_TABLE table=test-table [id=45e17a8841934f219ad9f2f8a6e1d0fb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 13:47:14.537093 15318 tablet_service.cc:1467] Processing CreateTablet for tablet fe2e7ae7153c4fe7a46830ffc8248a52 (DEFAULT_TABLE table=test-table [id=45e17a8841934f219ad9f2f8a6e1d0fb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 13:47:14.537396 15462 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fe2e7ae7153c4fe7a46830ffc8248a52. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 13:47:14.537396 15318 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fe2e7ae7153c4fe7a46830ffc8248a52. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 13:47:14.537353 15176 tablet_service.cc:1467] Processing CreateTablet for tablet fe2e7ae7153c4fe7a46830ffc8248a52 (DEFAULT_TABLE table=test-table [id=45e17a8841934f219ad9f2f8a6e1d0fb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 13:47:14.537698 15176 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fe2e7ae7153c4fe7a46830ffc8248a52. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 13:47:14.540984 15551 tablet_bootstrap.cc:492] T fe2e7ae7153c4fe7a46830ffc8248a52 P 2ab1deb934114642a088159e52c51b56: Bootstrap starting.
I20260924 13:47:14.541363 15552 tablet_bootstrap.cc:492] T fe2e7ae7153c4fe7a46830ffc8248a52 P 2d405e4bdb8141389defffbc73190a5b: Bootstrap starting.
I20260924 13:47:14.541786 15553 tablet_bootstrap.cc:492] T fe2e7ae7153c4fe7a46830ffc8248a52 P 21bfcfdb8dca41cca6a26000235952dd: Bootstrap starting.
I20260924 13:47:14.542021 15552 tablet_bootstrap.cc:654] T fe2e7ae7153c4fe7a46830ffc8248a52 P 2d405e4bdb8141389defffbc73190a5b: Neither blocks nor log segments found. Creating new log.
I20260924 13:47:14.542153 15551 tablet_bootstrap.cc:654] T fe2e7ae7153c4fe7a46830ffc8248a52 P 2ab1deb934114642a088159e52c51b56: Neither blocks nor log segments found. Creating new log.
I20260924 13:47:14.542372 15552 log.cc:824] T fe2e7ae7153c4fe7a46830ffc8248a52 P 2d405e4bdb8141389defffbc73190a5b: Log is configured to *not* fsync() on all Append() calls
I20260924 13:47:14.542493 15551 log.cc:824] T fe2e7ae7153c4fe7a46830ffc8248a52 P 2ab1deb934114642a088159e52c51b56: Log is configured to *not* fsync() on all Append() calls
I20260924 13:47:14.542610 15553 tablet_bootstrap.cc:654] T fe2e7ae7153c4fe7a46830ffc8248a52 P 21bfcfdb8dca41cca6a26000235952dd: Neither blocks nor log segments found. Creating new log.
I20260924 13:47:14.542949 15553 log.cc:824] T fe2e7ae7153c4fe7a46830ffc8248a52 P 21bfcfdb8dca41cca6a26000235952dd: Log is configured to *not* fsync() on all Append() calls
I20260924 13:47:14.543255 15551 tablet_bootstrap.cc:492] T fe2e7ae7153c4fe7a46830ffc8248a52 P 2ab1deb934114642a088159e52c51b56: No bootstrap required, opened a new log
I20260924 13:47:14.543253 15552 tablet_bootstrap.cc:492] T fe2e7ae7153c4fe7a46830ffc8248a52 P 2d405e4bdb8141389defffbc73190a5b: No bootstrap required, opened a new log
I20260924 13:47:14.543330 15552 ts_tablet_manager.cc:1397] T fe2e7ae7153c4fe7a46830ffc8248a52 P 2d405e4bdb8141389defffbc73190a5b: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260924 13:47:14.543330 15551 ts_tablet_manager.cc:1397] T fe2e7ae7153c4fe7a46830ffc8248a52 P 2ab1deb934114642a088159e52c51b56: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260924 13:47:14.543658 15553 tablet_bootstrap.cc:492] T fe2e7ae7153c4fe7a46830ffc8248a52 P 21bfcfdb8dca41cca6a26000235952dd: No bootstrap required, opened a new log
I20260924 13:47:14.543732 15553 ts_tablet_manager.cc:1397] T fe2e7ae7153c4fe7a46830ffc8248a52 P 21bfcfdb8dca41cca6a26000235952dd: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260924 13:47:14.545322 15553 raft_consensus.cc:348] T fe2e7ae7153c4fe7a46830ffc8248a52 P 21bfcfdb8dca41cca6a26000235952dd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d405e4bdb8141389defffbc73190a5b" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 36491 } } peers { permanent_uuid: "21bfcfdb8dca41cca6a26000235952dd" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 40107 } } peers { permanent_uuid: "2ab1deb934114642a088159e52c51b56" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 37565 } }
I20260924 13:47:14.545473 15553 raft_consensus.cc:374] T fe2e7ae7153c4fe7a46830ffc8248a52 P 21bfcfdb8dca41cca6a26000235952dd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 13:47:14.545504 15553 raft_consensus.cc:729] T fe2e7ae7153c4fe7a46830ffc8248a52 P 21bfcfdb8dca41cca6a26000235952dd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 21bfcfdb8dca41cca6a26000235952dd, State: Initialized, Role: FOLLOWER
I20260924 13:47:14.545500 15551 raft_consensus.cc:348] T fe2e7ae7153c4fe7a46830ffc8248a52 P 2ab1deb934114642a088159e52c51b56 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d405e4bdb8141389defffbc73190a5b" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 36491 } } peers { permanent_uuid: "21bfcfdb8dca41cca6a26000235952dd" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 40107 } } peers { permanent_uuid: "2ab1deb934114642a088159e52c51b56" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 37565 } }
I20260924 13:47:14.545500 15552 raft_consensus.cc:348] T fe2e7ae7153c4fe7a46830ffc8248a52 P 2d405e4bdb8141389defffbc73190a5b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d405e4bdb8141389defffbc73190a5b" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 36491 } } peers { permanent_uuid: "21bfcfdb8dca41cca6a26000235952dd" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 40107 } } peers { permanent_uuid: "2ab1deb934114642a088159e52c51b56" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 37565 } }
I20260924 13:47:14.545684 15552 raft_consensus.cc:374] T fe2e7ae7153c4fe7a46830ffc8248a52 P 2d405e4bdb8141389defffbc73190a5b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 13:47:14.545684 15551 raft_consensus.cc:374] T fe2e7ae7153c4fe7a46830ffc8248a52 P 2ab1deb934114642a088159e52c51b56 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 13:47:14.545658 15553 consensus_queue.cc:260] T fe2e7ae7153c4fe7a46830ffc8248a52 P 21bfcfdb8dca41cca6a26000235952dd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d405e4bdb8141389defffbc73190a5b" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 36491 } } peers { permanent_uuid: "21bfcfdb8dca41cca6a26000235952dd" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 40107 } } peers { permanent_uuid: "2ab1deb934114642a088159e52c51b56" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 37565 } }
I20260924 13:47:14.545729 15551 raft_consensus.cc:729] T fe2e7ae7153c4fe7a46830ffc8248a52 P 2ab1deb934114642a088159e52c51b56 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ab1deb934114642a088159e52c51b56, State: Initialized, Role: FOLLOWER
I20260924 13:47:14.545729 15552 raft_consensus.cc:729] T fe2e7ae7153c4fe7a46830ffc8248a52 P 2d405e4bdb8141389defffbc73190a5b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2d405e4bdb8141389defffbc73190a5b, State: Initialized, Role: FOLLOWER
I20260924 13:47:14.545848 15552 consensus_queue.cc:260] T fe2e7ae7153c4fe7a46830ffc8248a52 P 2d405e4bdb8141389defffbc73190a5b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d405e4bdb8141389defffbc73190a5b" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 36491 } } peers { permanent_uuid: "21bfcfdb8dca41cca6a26000235952dd" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 40107 } } peers { permanent_uuid: "2ab1deb934114642a088159e52c51b56" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 37565 } }
I20260924 13:47:14.545903 15551 consensus_queue.cc:260] T fe2e7ae7153c4fe7a46830ffc8248a52 P 2ab1deb934114642a088159e52c51b56 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d405e4bdb8141389defffbc73190a5b" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 36491 } } peers { permanent_uuid: "21bfcfdb8dca41cca6a26000235952dd" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 40107 } } peers { permanent_uuid: "2ab1deb934114642a088159e52c51b56" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 37565 } }
I20260924 13:47:14.545950 15553 ts_tablet_manager.cc:1428] T fe2e7ae7153c4fe7a46830ffc8248a52 P 21bfcfdb8dca41cca6a26000235952dd: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260924 13:47:14.546082 15242 heartbeater.cc:503] Master 127.13.149.254:35189 was elected leader, sending a full tablet report...
I20260924 13:47:14.546113 15552 ts_tablet_manager.cc:1428] T fe2e7ae7153c4fe7a46830ffc8248a52 P 2d405e4bdb8141389defffbc73190a5b: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260924 13:47:14.546113 15551 ts_tablet_manager.cc:1428] T fe2e7ae7153c4fe7a46830ffc8248a52 P 2ab1deb934114642a088159e52c51b56: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260924 13:47:14.546181 15387 heartbeater.cc:503] Master 127.13.149.254:35189 was elected leader, sending a full tablet report...
I20260924 13:47:14.546393 15528 heartbeater.cc:503] Master 127.13.149.254:35189 was elected leader, sending a full tablet report...
I20260924 13:47:14.546417 15558 raft_consensus.cc:482] T fe2e7ae7153c4fe7a46830ffc8248a52 P 2ab1deb934114642a088159e52c51b56 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 13:47:14.546938 15558 raft_consensus.cc:504] T fe2e7ae7153c4fe7a46830ffc8248a52 P 2ab1deb934114642a088159e52c51b56 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d405e4bdb8141389defffbc73190a5b" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 36491 } } peers { permanent_uuid: "21bfcfdb8dca41cca6a26000235952dd" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 40107 } } peers { permanent_uuid: "2ab1deb934114642a088159e52c51b56" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 37565 } }
I20260924 13:47:14.547263 15558 leader_election.cc:288] T fe2e7ae7153c4fe7a46830ffc8248a52 P 2ab1deb934114642a088159e52c51b56 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2d405e4bdb8141389defffbc73190a5b (127.13.149.194:36491), 21bfcfdb8dca41cca6a26000235952dd (127.13.149.193:40107)
I20260924 13:47:14.551514 15550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:14.547641 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.149.195:42835 (local address 127.13.149.193:40107)
0924 13:47:14.547692 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0924 13:47:14.547695 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 13:47:14.547866 (+ 171us) server_negotiation.cc:407] Connection header received
0924 13:47:14.547892 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:47:14.547892 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:47:14.547901 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:47:14.547943 (+ 42us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 13:47:14.548157 (+ 214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:14.548994 (+ 837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:14.550541 (+ 1547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:14.551139 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:14.551251 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:47:14.551260 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0924 13:47:14.551395 (+ 135us) server_negotiation.cc:299] Negotiation successful
0924 13:47:14.551434 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260924 13:47:14.551514 15547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:14.547757 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.149.195:47669 (local address 127.13.149.194:36491)
0924 13:47:14.547795 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0924 13:47:14.547797 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:47:14.547911 (+ 114us) server_negotiation.cc:407] Connection header received
0924 13:47:14.548000 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:47:14.548002 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:47:14.548011 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:47:14.548059 (+ 48us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 13:47:14.548186 (+ 127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:14.548961 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:14.550541 (+ 1580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:14.551131 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:14.551221 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:47:14.551233 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0924 13:47:14.551390 (+ 157us) server_negotiation.cc:299] Negotiation successful
0924 13:47:14.551434 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260924 13:47:14.551532 15420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:14.547575 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.193:40107 (local address 127.13.149.195:42835)
0924 13:47:14.547757 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0924 13:47:14.547768 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0924 13:47:14.547814 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:47:14.548020 (+ 206us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:47:14.548021 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:47:14.548023 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 13:47:14.548088 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:47:14.548089 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:47:14.549056 (+ 967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:47:14.549057 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:47:14.550394 (+ 1337us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:47:14.550398 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:47:14.551248 (+ 850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:47:14.551248 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:47:14.551330 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:47:14.551332 (+ 2us) client_negotiation.cc:769] Sending connection context
0924 13:47:14.551380 (+ 48us) client_negotiation.cc:240] Negotiation successful
0924 13:47:14.551433 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":167,"mutex_wait_us":70,"thread_start_us":87,"threads_started":1}
I20260924 13:47:14.551538 15561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:14.547479 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.194:36491 (local address 127.13.149.195:47669)
0924 13:47:14.547857 (+ 378us) negotiation.cc:107] Waiting for socket to connect
0924 13:47:14.547867 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 13:47:14.547920 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:47:14.548063 (+ 143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:47:14.548063 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:47:14.548065 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 13:47:14.548129 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:47:14.548130 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:47:14.549056 (+ 926us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:47:14.549057 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:47:14.550394 (+ 1337us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:47:14.550398 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:47:14.551226 (+ 828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:47:14.551227 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:47:14.551308 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:47:14.551314 (+ 6us) client_negotiation.cc:769] Sending connection context
0924 13:47:14.551380 (+ 66us) client_negotiation.cc:240] Negotiation successful
0924 13:47:14.551433 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":256,"mutex_wait_us":73}
I20260924 13:47:14.552199 15196 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe2e7ae7153c4fe7a46830ffc8248a52" candidate_uuid: "2ab1deb934114642a088159e52c51b56" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "21bfcfdb8dca41cca6a26000235952dd" is_pre_election: true
I20260924 13:47:14.552199 15339 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe2e7ae7153c4fe7a46830ffc8248a52" candidate_uuid: "2ab1deb934114642a088159e52c51b56" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d405e4bdb8141389defffbc73190a5b" is_pre_election: true
I20260924 13:47:14.552387 15196 raft_consensus.cc:2454] T fe2e7ae7153c4fe7a46830ffc8248a52 P 21bfcfdb8dca41cca6a26000235952dd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2ab1deb934114642a088159e52c51b56 in term 0.
I20260924 13:47:14.552387 15339 raft_consensus.cc:2454] T fe2e7ae7153c4fe7a46830ffc8248a52 P 2d405e4bdb8141389defffbc73190a5b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2ab1deb934114642a088159e52c51b56 in term 0.
I20260924 13:47:14.552654 15415 leader_election.cc:302] T fe2e7ae7153c4fe7a46830ffc8248a52 P 2ab1deb934114642a088159e52c51b56 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 21bfcfdb8dca41cca6a26000235952dd, 2ab1deb934114642a088159e52c51b56; no voters:
I20260924 13:47:14.552857 15558 raft_consensus.cc:2781] T fe2e7ae7153c4fe7a46830ffc8248a52 P 2ab1deb934114642a088159e52c51b56 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 13:47:14.552970 15558 raft_consensus.cc:482] T fe2e7ae7153c4fe7a46830ffc8248a52 P 2ab1deb934114642a088159e52c51b56 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 13:47:14.553005 15558 raft_consensus.cc:3037] T fe2e7ae7153c4fe7a46830ffc8248a52 P 2ab1deb934114642a088159e52c51b56 [term 0 FOLLOWER]: Advancing to term 1
I20260924 13:47:14.553757 15558 raft_consensus.cc:504] T fe2e7ae7153c4fe7a46830ffc8248a52 P 2ab1deb934114642a088159e52c51b56 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d405e4bdb8141389defffbc73190a5b" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 36491 } } peers { permanent_uuid: "21bfcfdb8dca41cca6a26000235952dd" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 40107 } } peers { permanent_uuid: "2ab1deb934114642a088159e52c51b56" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 37565 } }
I20260924 13:47:14.553911 15558 leader_election.cc:288] T fe2e7ae7153c4fe7a46830ffc8248a52 P 2ab1deb934114642a088159e52c51b56 [CANDIDATE]: Term 1 election: Requested vote from peers 2d405e4bdb8141389defffbc73190a5b (127.13.149.194:36491), 21bfcfdb8dca41cca6a26000235952dd (127.13.149.193:40107)
I20260924 13:47:14.554323 15339 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe2e7ae7153c4fe7a46830ffc8248a52" candidate_uuid: "2ab1deb934114642a088159e52c51b56" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d405e4bdb8141389defffbc73190a5b"
I20260924 13:47:14.554371 15196 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe2e7ae7153c4fe7a46830ffc8248a52" candidate_uuid: "2ab1deb934114642a088159e52c51b56" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "21bfcfdb8dca41cca6a26000235952dd"
I20260924 13:47:14.554507 15339 raft_consensus.cc:3037] T fe2e7ae7153c4fe7a46830ffc8248a52 P 2d405e4bdb8141389defffbc73190a5b [term 0 FOLLOWER]: Advancing to term 1
I20260924 13:47:14.554529 15196 raft_consensus.cc:3037] T fe2e7ae7153c4fe7a46830ffc8248a52 P 21bfcfdb8dca41cca6a26000235952dd [term 0 FOLLOWER]: Advancing to term 1
I20260924 13:47:14.555780 15196 raft_consensus.cc:2454] T fe2e7ae7153c4fe7a46830ffc8248a52 P 21bfcfdb8dca41cca6a26000235952dd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2ab1deb934114642a088159e52c51b56 in term 1.
I20260924 13:47:14.555780 15339 raft_consensus.cc:2454] T fe2e7ae7153c4fe7a46830ffc8248a52 P 2d405e4bdb8141389defffbc73190a5b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2ab1deb934114642a088159e52c51b56 in term 1.
I20260924 13:47:14.556094 15415 leader_election.cc:302] T fe2e7ae7153c4fe7a46830ffc8248a52 P 2ab1deb934114642a088159e52c51b56 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ab1deb934114642a088159e52c51b56, 2d405e4bdb8141389defffbc73190a5b; no voters:
I20260924 13:47:14.556324 15558 raft_consensus.cc:2781] T fe2e7ae7153c4fe7a46830ffc8248a52 P 2ab1deb934114642a088159e52c51b56 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 13:47:14.556432 15558 raft_consensus.cc:686] T fe2e7ae7153c4fe7a46830ffc8248a52 P 2ab1deb934114642a088159e52c51b56 [term 1 LEADER]: Becoming Leader. State: Replica: 2ab1deb934114642a088159e52c51b56, State: Running, Role: LEADER
I20260924 13:47:14.556563 15558 consensus_queue.cc:237] T fe2e7ae7153c4fe7a46830ffc8248a52 P 2ab1deb934114642a088159e52c51b56 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d405e4bdb8141389defffbc73190a5b" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 36491 } } peers { permanent_uuid: "21bfcfdb8dca41cca6a26000235952dd" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 40107 } } peers { permanent_uuid: "2ab1deb934114642a088159e52c51b56" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 37565 } }
I20260924 13:47:14.557798 15036 catalog_manager.cc:5495] T fe2e7ae7153c4fe7a46830ffc8248a52 P 2ab1deb934114642a088159e52c51b56 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2ab1deb934114642a088159e52c51b56 (127.13.149.195). New cstate: current_term: 1 leader_uuid: "2ab1deb934114642a088159e52c51b56" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d405e4bdb8141389defffbc73190a5b" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 36491 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "21bfcfdb8dca41cca6a26000235952dd" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 40107 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2ab1deb934114642a088159e52c51b56" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 37565 } health_report { overall_health: HEALTHY } } }
I20260924 13:47:14.569003 15549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:14.566271 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47664 (local address 127.13.149.195:37565)
0924 13:47:14.566323 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0924 13:47:14.566325 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:47:14.566333 (+ 8us) server_negotiation.cc:407] Connection header received
0924 13:47:14.566370 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:47:14.566371 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:47:14.566389 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:47:14.566453 (+ 64us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 13:47:14.566567 (+ 114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:14.567075 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:14.567961 (+ 886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:14.568157 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:14.568218 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:47:14.568503 (+ 285us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 13:47:14.568636 (+ 133us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 13:47:14.568714 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0924 13:47:14.568851 (+ 137us) server_negotiation.cc:299] Negotiation successful
0924 13:47:14.568876 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260924 13:47:14.571353 15196 raft_consensus.cc:1261] T fe2e7ae7153c4fe7a46830ffc8248a52 P 21bfcfdb8dca41cca6a26000235952dd [term 1 FOLLOWER]: Refusing update from remote peer 2ab1deb934114642a088159e52c51b56: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 13:47:14.571352 15339 raft_consensus.cc:1261] T fe2e7ae7153c4fe7a46830ffc8248a52 P 2d405e4bdb8141389defffbc73190a5b [term 1 FOLLOWER]: Refusing update from remote peer 2ab1deb934114642a088159e52c51b56: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 13:47:14.571794 15560 consensus_queue.cc:1035] T fe2e7ae7153c4fe7a46830ffc8248a52 P 2ab1deb934114642a088159e52c51b56 [LEADER]: Connected to new peer: Peer: permanent_uuid: "21bfcfdb8dca41cca6a26000235952dd" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 40107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 13:47:14.571973 15558 consensus_queue.cc:1035] T fe2e7ae7153c4fe7a46830ffc8248a52 P 2ab1deb934114642a088159e52c51b56 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2d405e4bdb8141389defffbc73190a5b" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 36491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 13:47:14.574645 15566 mvcc.cc:205] Tried to move back new op lower bound from 7332895271201443840 to 7332895271144747008. Current Snapshot: MvccSnapshot[applied={T|T < 7332895271201443840}]
I20260924 13:47:14.575074 15565 mvcc.cc:205] Tried to move back new op lower bound from 7332895271201443840 to 7332895271144747008. Current Snapshot: MvccSnapshot[applied={T|T < 7332895271201443840}]
I20260924 13:47:14.578174 15037 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44722:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 13:47:14.578270 15037 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44722:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 13:47:14.579025 15037 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 244154f1876f4f849fa9af203bd8fe2c: Sending DeleteTablet for 3 replicas of tablet fe2e7ae7153c4fe7a46830ffc8248a52
I20260924 13:47:14.579424 15176 tablet_service.cc:1514] Processing DeleteTablet for tablet fe2e7ae7153c4fe7a46830ffc8248a52 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 13:47:14 UTC) from {username='kudu', principal='kudu/127.13.149.254@KRBTEST.COM'} at 127.0.0.1:38352
I20260924 13:47:14.579432 15318 tablet_service.cc:1514] Processing DeleteTablet for tablet fe2e7ae7153c4fe7a46830ffc8248a52 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 13:47:14 UTC) from {username='kudu', principal='kudu/127.13.149.254@KRBTEST.COM'} at 127.0.0.1:41768
I20260924 13:47:14.579643 15037 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44722:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260924 13:47:14.579730 15462 tablet_service.cc:1514] Processing DeleteTablet for tablet fe2e7ae7153c4fe7a46830ffc8248a52 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 13:47:14 UTC) from {username='kudu', principal='kudu/127.13.149.254@KRBTEST.COM'} at 127.0.0.1:47654
W20260924 13:47:14.579806 15037 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 13:47:14.579941 15576 tablet_replica.cc:330] T fe2e7ae7153c4fe7a46830ffc8248a52 P 2ab1deb934114642a088159e52c51b56: stopping tablet replica
I20260924 13:47:14.580040 15576 raft_consensus.cc:2229] T fe2e7ae7153c4fe7a46830ffc8248a52 P 2ab1deb934114642a088159e52c51b56 [term 1 LEADER]: Raft consensus shutting down.
I20260924 13:47:14.580144 15576 raft_consensus.cc:2258] T fe2e7ae7153c4fe7a46830ffc8248a52 P 2ab1deb934114642a088159e52c51b56 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 13:47:14.580293 15574 tablet_replica.cc:330] T fe2e7ae7153c4fe7a46830ffc8248a52 P 21bfcfdb8dca41cca6a26000235952dd: stopping tablet replica
I20260924 13:47:14.580399 15574 raft_consensus.cc:2229] T fe2e7ae7153c4fe7a46830ffc8248a52 P 21bfcfdb8dca41cca6a26000235952dd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 13:47:14.580507 15574 raft_consensus.cc:2258] T fe2e7ae7153c4fe7a46830ffc8248a52 P 21bfcfdb8dca41cca6a26000235952dd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 13:47:14.581075 15576 ts_tablet_manager.cc:1905] T fe2e7ae7153c4fe7a46830ffc8248a52 P 2ab1deb934114642a088159e52c51b56: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 13:47:14.581209 15462 tablet_service.cc:1467] Processing CreateTablet for tablet c9909c5b7d014e389b30787c03d5dffd (DEFAULT_TABLE table=test-table [id=53908ce3aaa34a4892350cb4e13f76c5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 13:47:14.581313 15318 tablet_service.cc:1467] Processing CreateTablet for tablet c9909c5b7d014e389b30787c03d5dffd (DEFAULT_TABLE table=test-table [id=53908ce3aaa34a4892350cb4e13f76c5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 13:47:14.581423 15318 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c9909c5b7d014e389b30787c03d5dffd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 13:47:14.581672 15176 tablet_service.cc:1467] Processing CreateTablet for tablet c9909c5b7d014e389b30787c03d5dffd (DEFAULT_TABLE table=test-table [id=53908ce3aaa34a4892350cb4e13f76c5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 13:47:14.581769 15176 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c9909c5b7d014e389b30787c03d5dffd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 13:47:14.583148 15552 tablet_bootstrap.cc:492] T c9909c5b7d014e389b30787c03d5dffd P 2d405e4bdb8141389defffbc73190a5b: Bootstrap starting.
I20260924 13:47:14.583632 15552 tablet_bootstrap.cc:654] T c9909c5b7d014e389b30787c03d5dffd P 2d405e4bdb8141389defffbc73190a5b: Neither blocks nor log segments found. Creating new log.
I20260924 13:47:14.584014 15553 tablet_bootstrap.cc:492] T c9909c5b7d014e389b30787c03d5dffd P 21bfcfdb8dca41cca6a26000235952dd: Bootstrap starting.
I20260924 13:47:14.584266 15575 tablet_replica.cc:330] T fe2e7ae7153c4fe7a46830ffc8248a52 P 2d405e4bdb8141389defffbc73190a5b: stopping tablet replica
I20260924 13:47:14.584372 15552 tablet_bootstrap.cc:492] T c9909c5b7d014e389b30787c03d5dffd P 2d405e4bdb8141389defffbc73190a5b: No bootstrap required, opened a new log
I20260924 13:47:14.584389 15575 raft_consensus.cc:2229] T fe2e7ae7153c4fe7a46830ffc8248a52 P 2d405e4bdb8141389defffbc73190a5b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 13:47:14.584406 15552 ts_tablet_manager.cc:1397] T c9909c5b7d014e389b30787c03d5dffd P 2d405e4bdb8141389defffbc73190a5b: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260924 13:47:14.584435 15553 tablet_bootstrap.cc:654] T c9909c5b7d014e389b30787c03d5dffd P 21bfcfdb8dca41cca6a26000235952dd: Neither blocks nor log segments found. Creating new log.
I20260924 13:47:14.584529 15575 raft_consensus.cc:2258] T fe2e7ae7153c4fe7a46830ffc8248a52 P 2d405e4bdb8141389defffbc73190a5b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 13:47:14.584545 15552 raft_consensus.cc:348] T c9909c5b7d014e389b30787c03d5dffd P 2d405e4bdb8141389defffbc73190a5b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21bfcfdb8dca41cca6a26000235952dd" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 40107 } } peers { permanent_uuid: "2d405e4bdb8141389defffbc73190a5b" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 36491 } } peers { permanent_uuid: "2ab1deb934114642a088159e52c51b56" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 37565 } }
I20260924 13:47:14.584621 15552 raft_consensus.cc:374] T c9909c5b7d014e389b30787c03d5dffd P 2d405e4bdb8141389defffbc73190a5b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 13:47:14.584640 15552 raft_consensus.cc:729] T c9909c5b7d014e389b30787c03d5dffd P 2d405e4bdb8141389defffbc73190a5b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2d405e4bdb8141389defffbc73190a5b, State: Initialized, Role: FOLLOWER
I20260924 13:47:14.584681 15552 consensus_queue.cc:260] T c9909c5b7d014e389b30787c03d5dffd P 2d405e4bdb8141389defffbc73190a5b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21bfcfdb8dca41cca6a26000235952dd" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 40107 } } peers { permanent_uuid: "2d405e4bdb8141389defffbc73190a5b" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 36491 } } peers { permanent_uuid: "2ab1deb934114642a088159e52c51b56" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 37565 } }
I20260924 13:47:14.584755 15552 ts_tablet_manager.cc:1428] T c9909c5b7d014e389b30787c03d5dffd P 2d405e4bdb8141389defffbc73190a5b: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260924 13:47:14.584781 15462 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c9909c5b7d014e389b30787c03d5dffd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 13:47:14.585688 15576 ts_tablet_manager.cc:1918] T fe2e7ae7153c4fe7a46830ffc8248a52 P 2ab1deb934114642a088159e52c51b56: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 13:47:14.585774 15576 log.cc:1196] T fe2e7ae7153c4fe7a46830ffc8248a52 P 2ab1deb934114642a088159e52c51b56: Deleting WAL directory at /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790257629041645-13911-0/minicluster-data/ts-2/wal/wals/fe2e7ae7153c4fe7a46830ffc8248a52
I20260924 13:47:14.586026 15576 ts_tablet_manager.cc:1939] T fe2e7ae7153c4fe7a46830ffc8248a52 P 2ab1deb934114642a088159e52c51b56: Deleting consensus metadata
I20260924 13:47:14.586328 15551 tablet_bootstrap.cc:492] T c9909c5b7d014e389b30787c03d5dffd P 2ab1deb934114642a088159e52c51b56: Bootstrap starting.
I20260924 13:47:14.586400 15574 ts_tablet_manager.cc:1905] T fe2e7ae7153c4fe7a46830ffc8248a52 P 21bfcfdb8dca41cca6a26000235952dd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 13:47:14.586577 15024 catalog_manager.cc:4841] TS 2ab1deb934114642a088159e52c51b56 (127.13.149.195:37565): tablet fe2e7ae7153c4fe7a46830ffc8248a52 (table test-table [id=45e17a8841934f219ad9f2f8a6e1d0fb]) successfully deleted
I20260924 13:47:14.586726 15551 tablet_bootstrap.cc:654] T c9909c5b7d014e389b30787c03d5dffd P 2ab1deb934114642a088159e52c51b56: Neither blocks nor log segments found. Creating new log.
I20260924 13:47:14.587302 15551 tablet_bootstrap.cc:492] T c9909c5b7d014e389b30787c03d5dffd P 2ab1deb934114642a088159e52c51b56: No bootstrap required, opened a new log
I20260924 13:47:14.587344 15551 ts_tablet_manager.cc:1397] T c9909c5b7d014e389b30787c03d5dffd P 2ab1deb934114642a088159e52c51b56: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260924 13:47:14.587458 15551 raft_consensus.cc:348] T c9909c5b7d014e389b30787c03d5dffd P 2ab1deb934114642a088159e52c51b56 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21bfcfdb8dca41cca6a26000235952dd" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 40107 } } peers { permanent_uuid: "2d405e4bdb8141389defffbc73190a5b" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 36491 } } peers { permanent_uuid: "2ab1deb934114642a088159e52c51b56" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 37565 } }
I20260924 13:47:14.587515 15551 raft_consensus.cc:374] T c9909c5b7d014e389b30787c03d5dffd P 2ab1deb934114642a088159e52c51b56 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 13:47:14.587527 15551 raft_consensus.cc:729] T c9909c5b7d014e389b30787c03d5dffd P 2ab1deb934114642a088159e52c51b56 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ab1deb934114642a088159e52c51b56, State: Initialized, Role: FOLLOWER
I20260924 13:47:14.587563 15551 consensus_queue.cc:260] T c9909c5b7d014e389b30787c03d5dffd P 2ab1deb934114642a088159e52c51b56 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21bfcfdb8dca41cca6a26000235952dd" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 40107 } } peers { permanent_uuid: "2d405e4bdb8141389defffbc73190a5b" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 36491 } } peers { permanent_uuid: "2ab1deb934114642a088159e52c51b56" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 37565 } }
I20260924 13:47:14.587643 15551 ts_tablet_manager.cc:1428] T c9909c5b7d014e389b30787c03d5dffd P 2ab1deb934114642a088159e52c51b56: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260924 13:47:14.587767 15574 ts_tablet_manager.cc:1918] T fe2e7ae7153c4fe7a46830ffc8248a52 P 21bfcfdb8dca41cca6a26000235952dd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 13:47:14.587841 15574 log.cc:1196] T fe2e7ae7153c4fe7a46830ffc8248a52 P 21bfcfdb8dca41cca6a26000235952dd: Deleting WAL directory at /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790257629041645-13911-0/minicluster-data/ts-0/wal/wals/fe2e7ae7153c4fe7a46830ffc8248a52
I20260924 13:47:14.588076 15574 ts_tablet_manager.cc:1939] T fe2e7ae7153c4fe7a46830ffc8248a52 P 21bfcfdb8dca41cca6a26000235952dd: Deleting consensus metadata
I20260924 13:47:14.588471 15024 catalog_manager.cc:4841] TS 21bfcfdb8dca41cca6a26000235952dd (127.13.149.193:40107): tablet fe2e7ae7153c4fe7a46830ffc8248a52 (table test-table [id=45e17a8841934f219ad9f2f8a6e1d0fb]) successfully deleted
I20260924 13:47:14.588944 15575 ts_tablet_manager.cc:1905] T fe2e7ae7153c4fe7a46830ffc8248a52 P 2d405e4bdb8141389defffbc73190a5b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 13:47:14.589401 15553 tablet_bootstrap.cc:492] T c9909c5b7d014e389b30787c03d5dffd P 21bfcfdb8dca41cca6a26000235952dd: No bootstrap required, opened a new log
I20260924 13:47:14.589460 15553 ts_tablet_manager.cc:1397] T c9909c5b7d014e389b30787c03d5dffd P 21bfcfdb8dca41cca6a26000235952dd: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20260924 13:47:14.589612 15553 raft_consensus.cc:348] T c9909c5b7d014e389b30787c03d5dffd P 21bfcfdb8dca41cca6a26000235952dd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21bfcfdb8dca41cca6a26000235952dd" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 40107 } } peers { permanent_uuid: "2d405e4bdb8141389defffbc73190a5b" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 36491 } } peers { permanent_uuid: "2ab1deb934114642a088159e52c51b56" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 37565 } }
I20260924 13:47:14.589696 15553 raft_consensus.cc:374] T c9909c5b7d014e389b30787c03d5dffd P 21bfcfdb8dca41cca6a26000235952dd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 13:47:14.589715 15553 raft_consensus.cc:729] T c9909c5b7d014e389b30787c03d5dffd P 21bfcfdb8dca41cca6a26000235952dd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 21bfcfdb8dca41cca6a26000235952dd, State: Initialized, Role: FOLLOWER
I20260924 13:47:14.589756 15553 consensus_queue.cc:260] T c9909c5b7d014e389b30787c03d5dffd P 21bfcfdb8dca41cca6a26000235952dd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21bfcfdb8dca41cca6a26000235952dd" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 40107 } } peers { permanent_uuid: "2d405e4bdb8141389defffbc73190a5b" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 36491 } } peers { permanent_uuid: "2ab1deb934114642a088159e52c51b56" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 37565 } }
I20260924 13:47:14.589844 15553 ts_tablet_manager.cc:1428] T c9909c5b7d014e389b30787c03d5dffd P 21bfcfdb8dca41cca6a26000235952dd: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260924 13:47:14.591486 15575 ts_tablet_manager.cc:1918] T fe2e7ae7153c4fe7a46830ffc8248a52 P 2d405e4bdb8141389defffbc73190a5b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 13:47:14.591578 15575 log.cc:1196] T fe2e7ae7153c4fe7a46830ffc8248a52 P 2d405e4bdb8141389defffbc73190a5b: Deleting WAL directory at /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790257629041645-13911-0/minicluster-data/ts-1/wal/wals/fe2e7ae7153c4fe7a46830ffc8248a52
I20260924 13:47:14.591849 15575 ts_tablet_manager.cc:1939] T fe2e7ae7153c4fe7a46830ffc8248a52 P 2d405e4bdb8141389defffbc73190a5b: Deleting consensus metadata
I20260924 13:47:14.592294 15024 catalog_manager.cc:4841] TS 2d405e4bdb8141389defffbc73190a5b (127.13.149.194:36491): tablet fe2e7ae7153c4fe7a46830ffc8248a52 (table test-table [id=45e17a8841934f219ad9f2f8a6e1d0fb]) successfully deleted
W20260924 13:47:14.671679 15243 tablet.cc:2369] T c9909c5b7d014e389b30787c03d5dffd P 21bfcfdb8dca41cca6a26000235952dd: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260924 13:47:14.740875 15529 tablet.cc:2369] T c9909c5b7d014e389b30787c03d5dffd P 2ab1deb934114642a088159e52c51b56: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260924 13:47:14.809840 15559 raft_consensus.cc:482] T c9909c5b7d014e389b30787c03d5dffd P 2d405e4bdb8141389defffbc73190a5b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 13:47:14.809973 15559 raft_consensus.cc:504] T c9909c5b7d014e389b30787c03d5dffd P 2d405e4bdb8141389defffbc73190a5b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21bfcfdb8dca41cca6a26000235952dd" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 40107 } } peers { permanent_uuid: "2d405e4bdb8141389defffbc73190a5b" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 36491 } } peers { permanent_uuid: "2ab1deb934114642a088159e52c51b56" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 37565 } }
I20260924 13:47:14.810343 15559 leader_election.cc:288] T c9909c5b7d014e389b30787c03d5dffd P 2d405e4bdb8141389defffbc73190a5b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 21bfcfdb8dca41cca6a26000235952dd (127.13.149.193:40107), 2ab1deb934114642a088159e52c51b56 (127.13.149.195:37565)
I20260924 13:47:14.814102 15275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:14.810488 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.193:40107 (local address 127.13.149.194:59289)
0924 13:47:14.810553 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0924 13:47:14.810567 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0924 13:47:14.810636 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:47:14.810861 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:47:14.810862 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:47:14.810864 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 13:47:14.810961 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:47:14.810963 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:47:14.811873 (+ 910us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:47:14.811874 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:47:14.813087 (+ 1213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:47:14.813091 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:47:14.813849 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:47:14.813850 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:47:14.813920 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:47:14.813925 (+ 5us) client_negotiation.cc:769] Sending connection context
0924 13:47:14.813974 (+ 49us) client_negotiation.cc:240] Negotiation successful
0924 13:47:14.814011 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44}
I20260924 13:47:14.814114 15550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:14.810554 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.149.194:59289 (local address 127.13.149.193:40107)
0924 13:47:14.810600 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0924 13:47:14.810602 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:47:14.810619 (+ 17us) server_negotiation.cc:407] Connection header received
0924 13:47:14.810720 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:47:14.810721 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:47:14.810730 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:47:14.810776 (+ 46us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 13:47:14.811057 (+ 281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:14.811801 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:14.813199 (+ 1398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:14.813784 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:14.813848 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:47:14.813855 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0924 13:47:14.813999 (+ 144us) server_negotiation.cc:299] Negotiation successful
0924 13:47:14.814044 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260924 13:47:14.814167 15549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:14.810740 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.149.194:48297 (local address 127.13.149.195:37565)
0924 13:47:14.810808 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0924 13:47:14.810810 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:47:14.810822 (+ 12us) server_negotiation.cc:407] Connection header received
0924 13:47:14.810846 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:47:14.810847 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:47:14.810860 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:47:14.810910 (+ 50us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 13:47:14.811062 (+ 152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:14.811793 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:14.813213 (+ 1420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:14.813779 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:14.813830 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:47:14.813840 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0924 13:47:14.814056 (+ 216us) server_negotiation.cc:299] Negotiation successful
0924 13:47:14.814107 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260924 13:47:14.814376 15580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:14.810581 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.195:37565 (local address 127.13.149.194:48297)
0924 13:47:14.810700 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0924 13:47:14.810710 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 13:47:14.810759 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:47:14.810925 (+ 166us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:47:14.810925 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:47:14.810927 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 13:47:14.810981 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:47:14.810982 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:47:14.811892 (+ 910us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:47:14.811892 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:47:14.813108 (+ 1216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:47:14.813112 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:47:14.813897 (+ 785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:47:14.813897 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:47:14.813977 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:47:14.813980 (+ 3us) client_negotiation.cc:769] Sending connection context
0924 13:47:14.814031 (+ 51us) client_negotiation.cc:240] Negotiation successful
0924 13:47:14.814070 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":85,"threads_started":1}
I20260924 13:47:14.814479 15196 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c9909c5b7d014e389b30787c03d5dffd" candidate_uuid: "2d405e4bdb8141389defffbc73190a5b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "21bfcfdb8dca41cca6a26000235952dd" is_pre_election: true
I20260924 13:47:14.814579 15196 raft_consensus.cc:2454] T c9909c5b7d014e389b30787c03d5dffd P 21bfcfdb8dca41cca6a26000235952dd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2d405e4bdb8141389defffbc73190a5b in term 0.
I20260924 13:47:14.814600 15482 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c9909c5b7d014e389b30787c03d5dffd" candidate_uuid: "2d405e4bdb8141389defffbc73190a5b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ab1deb934114642a088159e52c51b56" is_pre_election: true
I20260924 13:47:14.814749 15482 raft_consensus.cc:2454] T c9909c5b7d014e389b30787c03d5dffd P 2ab1deb934114642a088159e52c51b56 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2d405e4bdb8141389defffbc73190a5b in term 0.
I20260924 13:47:14.814749 15270 leader_election.cc:302] T c9909c5b7d014e389b30787c03d5dffd P 2d405e4bdb8141389defffbc73190a5b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 21bfcfdb8dca41cca6a26000235952dd, 2d405e4bdb8141389defffbc73190a5b; no voters:
I20260924 13:47:14.814875 15559 raft_consensus.cc:2781] T c9909c5b7d014e389b30787c03d5dffd P 2d405e4bdb8141389defffbc73190a5b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 13:47:14.814930 15559 raft_consensus.cc:482] T c9909c5b7d014e389b30787c03d5dffd P 2d405e4bdb8141389defffbc73190a5b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 13:47:14.814960 15559 raft_consensus.cc:3037] T c9909c5b7d014e389b30787c03d5dffd P 2d405e4bdb8141389defffbc73190a5b [term 0 FOLLOWER]: Advancing to term 1
I20260924 13:47:14.815627 15559 raft_consensus.cc:504] T c9909c5b7d014e389b30787c03d5dffd P 2d405e4bdb8141389defffbc73190a5b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21bfcfdb8dca41cca6a26000235952dd" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 40107 } } peers { permanent_uuid: "2d405e4bdb8141389defffbc73190a5b" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 36491 } } peers { permanent_uuid: "2ab1deb934114642a088159e52c51b56" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 37565 } }
I20260924 13:47:14.815762 15559 leader_election.cc:288] T c9909c5b7d014e389b30787c03d5dffd P 2d405e4bdb8141389defffbc73190a5b [CANDIDATE]: Term 1 election: Requested vote from peers 21bfcfdb8dca41cca6a26000235952dd (127.13.149.193:40107), 2ab1deb934114642a088159e52c51b56 (127.13.149.195:37565)
I20260924 13:47:14.815938 15482 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c9909c5b7d014e389b30787c03d5dffd" candidate_uuid: "2d405e4bdb8141389defffbc73190a5b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ab1deb934114642a088159e52c51b56"
I20260924 13:47:14.815954 15196 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c9909c5b7d014e389b30787c03d5dffd" candidate_uuid: "2d405e4bdb8141389defffbc73190a5b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "21bfcfdb8dca41cca6a26000235952dd"
I20260924 13:47:14.816015 15482 raft_consensus.cc:3037] T c9909c5b7d014e389b30787c03d5dffd P 2ab1deb934114642a088159e52c51b56 [term 0 FOLLOWER]: Advancing to term 1
I20260924 13:47:14.816049 15196 raft_consensus.cc:3037] T c9909c5b7d014e389b30787c03d5dffd P 21bfcfdb8dca41cca6a26000235952dd [term 0 FOLLOWER]: Advancing to term 1
I20260924 13:47:14.816649 15482 raft_consensus.cc:2454] T c9909c5b7d014e389b30787c03d5dffd P 2ab1deb934114642a088159e52c51b56 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2d405e4bdb8141389defffbc73190a5b in term 1.
I20260924 13:47:14.816716 15196 raft_consensus.cc:2454] T c9909c5b7d014e389b30787c03d5dffd P 21bfcfdb8dca41cca6a26000235952dd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2d405e4bdb8141389defffbc73190a5b in term 1.
I20260924 13:47:14.816855 15270 leader_election.cc:302] T c9909c5b7d014e389b30787c03d5dffd P 2d405e4bdb8141389defffbc73190a5b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ab1deb934114642a088159e52c51b56, 2d405e4bdb8141389defffbc73190a5b; no voters:
I20260924 13:47:14.816982 15559 raft_consensus.cc:2781] T c9909c5b7d014e389b30787c03d5dffd P 2d405e4bdb8141389defffbc73190a5b [term 1 FOLLOWER]: Leader election won for term 1
I20260924 13:47:14.817099 15559 raft_consensus.cc:686] T c9909c5b7d014e389b30787c03d5dffd P 2d405e4bdb8141389defffbc73190a5b [term 1 LEADER]: Becoming Leader. State: Replica: 2d405e4bdb8141389defffbc73190a5b, State: Running, Role: LEADER
I20260924 13:47:14.817169 15559 consensus_queue.cc:237] T c9909c5b7d014e389b30787c03d5dffd P 2d405e4bdb8141389defffbc73190a5b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21bfcfdb8dca41cca6a26000235952dd" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 40107 } } peers { permanent_uuid: "2d405e4bdb8141389defffbc73190a5b" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 36491 } } peers { permanent_uuid: "2ab1deb934114642a088159e52c51b56" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 37565 } }
I20260924 13:47:14.817809 15037 catalog_manager.cc:5495] T c9909c5b7d014e389b30787c03d5dffd P 2d405e4bdb8141389defffbc73190a5b reported cstate change: term changed from 0 to 1, leader changed from <none> to 2d405e4bdb8141389defffbc73190a5b (127.13.149.194). New cstate: current_term: 1 leader_uuid: "2d405e4bdb8141389defffbc73190a5b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21bfcfdb8dca41cca6a26000235952dd" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 40107 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2d405e4bdb8141389defffbc73190a5b" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 36491 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2ab1deb934114642a088159e52c51b56" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 37565 } health_report { overall_health: UNKNOWN } } }
W20260924 13:47:14.834941 15388 tablet.cc:2369] T c9909c5b7d014e389b30787c03d5dffd P 2d405e4bdb8141389defffbc73190a5b: Can't schedule compaction. Clean time has not been advanced past its initial value.
Sep 24 13:47:14 dist-test-slave-sjl2 krb5kdc[14992](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257633, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.149.254@KRBTEST.COM for kudu/127.13.149.254@KRBTEST.COM
I20260924 13:47:14.893275 15588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:14.885761 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:35189 (local address 127.0.0.1:44738)
0924 13:47:14.885965 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0924 13:47:14.885976 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0924 13:47:14.886045 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:47:14.886193 (+ 148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:47:14.886194 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:47:14.886512 (+ 318us) client_negotiation.cc:189] Negotiated authn=SASL
0924 13:47:14.886563 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:47:14.886566 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:47:14.887272 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:47:14.887273 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:47:14.887832 (+ 559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:47:14.887834 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:47:14.888214 (+ 380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:47:14.888215 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:47:14.888268 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:47:14.888891 (+ 623us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 13:47:14.888896 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 13:47:14.890553 (+ 1657us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 13:47:14.892163 (+ 1610us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 13:47:14.892176 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 13:47:14.892179 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0924 13:47:14.892424 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 13:47:14.892640 (+ 216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 13:47:14.892641 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 13:47:14.892641 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 13:47:14.892695 (+ 54us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 13:47:14.892906 (+ 211us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 13:47:14.892912 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 13:47:14.893006 (+ 94us) client_negotiation.cc:769] Sending connection context
0924 13:47:14.893061 (+ 55us) client_negotiation.cc:240] Negotiation successful
0924 13:47:14.893174 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":95,"threads_started":1}
I20260924 13:47:14.893321 15088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:14.885869 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44738 (local address 127.13.149.254:35189)
0924 13:47:14.885935 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0924 13:47:14.885937 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:47:14.886038 (+ 101us) server_negotiation.cc:407] Connection header received
0924 13:47:14.886088 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:47:14.886088 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:47:14.886098 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:47:14.886151 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:47:14.886666 (+ 515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:14.887190 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:14.887926 (+ 736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:14.888117 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:14.888180 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:47:14.890663 (+ 2483us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:47:14.890664 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:47:14.890664 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 13:47:14.890672 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:47:14.892054 (+ 1382us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:47:14.892518 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:47:14.892519 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:47:14.892519 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:47:14.892579 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:47:14.892739 (+ 160us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:47:14.892740 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:47:14.892740 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:47:14.892858 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:47:14.892908 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0924 13:47:14.893086 (+ 178us) server_negotiation.cc:299] Negotiation successful
0924 13:47:14.893227 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":64,"server-negotiator.queue_time_us":40}
I20260924 13:47:14.896111 15037 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.13.149.254@KRBTEST.COM'} at 127.0.0.1:44738:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260924 13:47:14.896376 15037 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 13:47:14.897858 15176 tablet_service.cc:1467] Processing CreateTablet for tablet b3495228ff2748f6afd56e75c1ad6ede (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=0cbdc71b4b424117b94cbbd58b9998be]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260924 13:47:14.897917 15318 tablet_service.cc:1467] Processing CreateTablet for tablet b3495228ff2748f6afd56e75c1ad6ede (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=0cbdc71b4b424117b94cbbd58b9998be]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260924 13:47:14.897919 15462 tablet_service.cc:1467] Processing CreateTablet for tablet b3495228ff2748f6afd56e75c1ad6ede (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=0cbdc71b4b424117b94cbbd58b9998be]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260924 13:47:14.898051 15176 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b3495228ff2748f6afd56e75c1ad6ede. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 13:47:14.898051 15318 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b3495228ff2748f6afd56e75c1ad6ede. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 13:47:14.898069 15462 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b3495228ff2748f6afd56e75c1ad6ede. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 13:47:14.899415 15551 tablet_bootstrap.cc:492] T b3495228ff2748f6afd56e75c1ad6ede P 2ab1deb934114642a088159e52c51b56: Bootstrap starting.
I20260924 13:47:14.899603 15553 tablet_bootstrap.cc:492] T b3495228ff2748f6afd56e75c1ad6ede P 21bfcfdb8dca41cca6a26000235952dd: Bootstrap starting.
I20260924 13:47:14.899618 15552 tablet_bootstrap.cc:492] T b3495228ff2748f6afd56e75c1ad6ede P 2d405e4bdb8141389defffbc73190a5b: Bootstrap starting.
I20260924 13:47:14.899775 15551 tablet_bootstrap.cc:654] T b3495228ff2748f6afd56e75c1ad6ede P 2ab1deb934114642a088159e52c51b56: Neither blocks nor log segments found. Creating new log.
I20260924 13:47:14.899993 15553 tablet_bootstrap.cc:654] T b3495228ff2748f6afd56e75c1ad6ede P 21bfcfdb8dca41cca6a26000235952dd: Neither blocks nor log segments found. Creating new log.
I20260924 13:47:14.900007 15552 tablet_bootstrap.cc:654] T b3495228ff2748f6afd56e75c1ad6ede P 2d405e4bdb8141389defffbc73190a5b: Neither blocks nor log segments found. Creating new log.
I20260924 13:47:14.900581 15553 tablet_bootstrap.cc:492] T b3495228ff2748f6afd56e75c1ad6ede P 21bfcfdb8dca41cca6a26000235952dd: No bootstrap required, opened a new log
I20260924 13:47:14.900640 15553 ts_tablet_manager.cc:1397] T b3495228ff2748f6afd56e75c1ad6ede P 21bfcfdb8dca41cca6a26000235952dd: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260924 13:47:14.900672 15552 tablet_bootstrap.cc:492] T b3495228ff2748f6afd56e75c1ad6ede P 2d405e4bdb8141389defffbc73190a5b: No bootstrap required, opened a new log
I20260924 13:47:14.900681 15551 tablet_bootstrap.cc:492] T b3495228ff2748f6afd56e75c1ad6ede P 2ab1deb934114642a088159e52c51b56: No bootstrap required, opened a new log
I20260924 13:47:14.900717 15552 ts_tablet_manager.cc:1397] T b3495228ff2748f6afd56e75c1ad6ede P 2d405e4bdb8141389defffbc73190a5b: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260924 13:47:14.900720 15551 ts_tablet_manager.cc:1397] T b3495228ff2748f6afd56e75c1ad6ede P 2ab1deb934114642a088159e52c51b56: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260924 13:47:14.900833 15551 raft_consensus.cc:348] T b3495228ff2748f6afd56e75c1ad6ede P 2ab1deb934114642a088159e52c51b56 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21bfcfdb8dca41cca6a26000235952dd" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 40107 } } peers { permanent_uuid: "2d405e4bdb8141389defffbc73190a5b" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 36491 } } peers { permanent_uuid: "2ab1deb934114642a088159e52c51b56" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 37565 } }
I20260924 13:47:14.900816 15553 raft_consensus.cc:348] T b3495228ff2748f6afd56e75c1ad6ede P 21bfcfdb8dca41cca6a26000235952dd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21bfcfdb8dca41cca6a26000235952dd" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 40107 } } peers { permanent_uuid: "2d405e4bdb8141389defffbc73190a5b" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 36491 } } peers { permanent_uuid: "2ab1deb934114642a088159e52c51b56" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 37565 } }
I20260924 13:47:14.900902 15551 raft_consensus.cc:374] T b3495228ff2748f6afd56e75c1ad6ede P 2ab1deb934114642a088159e52c51b56 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 13:47:14.900902 15553 raft_consensus.cc:374] T b3495228ff2748f6afd56e75c1ad6ede P 21bfcfdb8dca41cca6a26000235952dd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 13:47:14.900883 15552 raft_consensus.cc:348] T b3495228ff2748f6afd56e75c1ad6ede P 2d405e4bdb8141389defffbc73190a5b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21bfcfdb8dca41cca6a26000235952dd" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 40107 } } peers { permanent_uuid: "2d405e4bdb8141389defffbc73190a5b" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 36491 } } peers { permanent_uuid: "2ab1deb934114642a088159e52c51b56" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 37565 } }
I20260924 13:47:14.900957 15551 raft_consensus.cc:729] T b3495228ff2748f6afd56e75c1ad6ede P 2ab1deb934114642a088159e52c51b56 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ab1deb934114642a088159e52c51b56, State: Initialized, Role: FOLLOWER
I20260924 13:47:14.900957 15553 raft_consensus.cc:729] T b3495228ff2748f6afd56e75c1ad6ede P 21bfcfdb8dca41cca6a26000235952dd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 21bfcfdb8dca41cca6a26000235952dd, State: Initialized, Role: FOLLOWER
I20260924 13:47:14.900970 15552 raft_consensus.cc:374] T b3495228ff2748f6afd56e75c1ad6ede P 2d405e4bdb8141389defffbc73190a5b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 13:47:14.900995 15552 raft_consensus.cc:729] T b3495228ff2748f6afd56e75c1ad6ede P 2d405e4bdb8141389defffbc73190a5b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2d405e4bdb8141389defffbc73190a5b, State: Initialized, Role: FOLLOWER
I20260924 13:47:14.901006 15553 consensus_queue.cc:260] T b3495228ff2748f6afd56e75c1ad6ede P 21bfcfdb8dca41cca6a26000235952dd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21bfcfdb8dca41cca6a26000235952dd" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 40107 } } peers { permanent_uuid: "2d405e4bdb8141389defffbc73190a5b" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 36491 } } peers { permanent_uuid: "2ab1deb934114642a088159e52c51b56" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 37565 } }
I20260924 13:47:14.901006 15551 consensus_queue.cc:260] T b3495228ff2748f6afd56e75c1ad6ede P 2ab1deb934114642a088159e52c51b56 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21bfcfdb8dca41cca6a26000235952dd" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 40107 } } peers { permanent_uuid: "2d405e4bdb8141389defffbc73190a5b" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 36491 } } peers { permanent_uuid: "2ab1deb934114642a088159e52c51b56" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 37565 } }
I20260924 13:47:14.901034 15552 consensus_queue.cc:260] T b3495228ff2748f6afd56e75c1ad6ede P 2d405e4bdb8141389defffbc73190a5b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21bfcfdb8dca41cca6a26000235952dd" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 40107 } } peers { permanent_uuid: "2d405e4bdb8141389defffbc73190a5b" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 36491 } } peers { permanent_uuid: "2ab1deb934114642a088159e52c51b56" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 37565 } }
I20260924 13:47:14.901114 15552 ts_tablet_manager.cc:1428] T b3495228ff2748f6afd56e75c1ad6ede P 2d405e4bdb8141389defffbc73190a5b: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260924 13:47:14.901119 15553 ts_tablet_manager.cc:1428] T b3495228ff2748f6afd56e75c1ad6ede P 21bfcfdb8dca41cca6a26000235952dd: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260924 13:47:14.901120 15551 ts_tablet_manager.cc:1428] T b3495228ff2748f6afd56e75c1ad6ede P 2ab1deb934114642a088159e52c51b56: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260924 13:47:14.901227 15559 tablet_replica.cc:439] T b3495228ff2748f6afd56e75c1ad6ede P 2d405e4bdb8141389defffbc73190a5b: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21bfcfdb8dca41cca6a26000235952dd" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 40107 } } peers { permanent_uuid: "2d405e4bdb8141389defffbc73190a5b" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 36491 } } peers { permanent_uuid: "2ab1deb934114642a088159e52c51b56" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 37565 } } }
I20260924 13:47:14.901221 15569 tablet_replica.cc:439] T b3495228ff2748f6afd56e75c1ad6ede P 2ab1deb934114642a088159e52c51b56: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21bfcfdb8dca41cca6a26000235952dd" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 40107 } } peers { permanent_uuid: "2d405e4bdb8141389defffbc73190a5b" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 36491 } } peers { permanent_uuid: "2ab1deb934114642a088159e52c51b56" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 37565 } } }
I20260924 13:47:14.901283 15559 tablet_replica.cc:442] T b3495228ff2748f6afd56e75c1ad6ede P 2d405e4bdb8141389defffbc73190a5b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260924 13:47:14.901259 15557 tablet_replica.cc:439] T b3495228ff2748f6afd56e75c1ad6ede P 21bfcfdb8dca41cca6a26000235952dd: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21bfcfdb8dca41cca6a26000235952dd" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 40107 } } peers { permanent_uuid: "2d405e4bdb8141389defffbc73190a5b" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 36491 } } peers { permanent_uuid: "2ab1deb934114642a088159e52c51b56" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 37565 } } }
I20260924 13:47:14.901301 15569 tablet_replica.cc:442] T b3495228ff2748f6afd56e75c1ad6ede P 2ab1deb934114642a088159e52c51b56: This TxnStatusTablet replica's current role is: FOLLOWER
I20260924 13:47:14.901335 15557 tablet_replica.cc:442] T b3495228ff2748f6afd56e75c1ad6ede P 21bfcfdb8dca41cca6a26000235952dd: This TxnStatusTablet replica's current role is: FOLLOWER
I20260924 13:47:15.021757 15559 raft_consensus.cc:482] T b3495228ff2748f6afd56e75c1ad6ede P 2d405e4bdb8141389defffbc73190a5b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 13:47:15.021871 15559 raft_consensus.cc:504] T b3495228ff2748f6afd56e75c1ad6ede P 2d405e4bdb8141389defffbc73190a5b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21bfcfdb8dca41cca6a26000235952dd" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 40107 } } peers { permanent_uuid: "2d405e4bdb8141389defffbc73190a5b" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 36491 } } peers { permanent_uuid: "2ab1deb934114642a088159e52c51b56" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 37565 } }
I20260924 13:47:15.022013 15559 leader_election.cc:288] T b3495228ff2748f6afd56e75c1ad6ede P 2d405e4bdb8141389defffbc73190a5b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 21bfcfdb8dca41cca6a26000235952dd (127.13.149.193:40107), 2ab1deb934114642a088159e52c51b56 (127.13.149.195:37565)
I20260924 13:47:15.022334 15482 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b3495228ff2748f6afd56e75c1ad6ede" candidate_uuid: "2d405e4bdb8141389defffbc73190a5b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ab1deb934114642a088159e52c51b56" is_pre_election: true
I20260924 13:47:15.022343 15196 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b3495228ff2748f6afd56e75c1ad6ede" candidate_uuid: "2d405e4bdb8141389defffbc73190a5b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "21bfcfdb8dca41cca6a26000235952dd" is_pre_election: true
I20260924 13:47:15.022418 15482 raft_consensus.cc:2454] T b3495228ff2748f6afd56e75c1ad6ede P 2ab1deb934114642a088159e52c51b56 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2d405e4bdb8141389defffbc73190a5b in term 0.
I20260924 13:47:15.022445 15196 raft_consensus.cc:2454] T b3495228ff2748f6afd56e75c1ad6ede P 21bfcfdb8dca41cca6a26000235952dd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2d405e4bdb8141389defffbc73190a5b in term 0.
I20260924 13:47:15.022568 15270 leader_election.cc:302] T b3495228ff2748f6afd56e75c1ad6ede P 2d405e4bdb8141389defffbc73190a5b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ab1deb934114642a088159e52c51b56, 2d405e4bdb8141389defffbc73190a5b; no voters:
I20260924 13:47:15.022662 15559 raft_consensus.cc:2781] T b3495228ff2748f6afd56e75c1ad6ede P 2d405e4bdb8141389defffbc73190a5b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 13:47:15.022703 15559 raft_consensus.cc:482] T b3495228ff2748f6afd56e75c1ad6ede P 2d405e4bdb8141389defffbc73190a5b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 13:47:15.022727 15559 raft_consensus.cc:3037] T b3495228ff2748f6afd56e75c1ad6ede P 2d405e4bdb8141389defffbc73190a5b [term 0 FOLLOWER]: Advancing to term 1
I20260924 13:47:15.023480 15559 raft_consensus.cc:504] T b3495228ff2748f6afd56e75c1ad6ede P 2d405e4bdb8141389defffbc73190a5b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21bfcfdb8dca41cca6a26000235952dd" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 40107 } } peers { permanent_uuid: "2d405e4bdb8141389defffbc73190a5b" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 36491 } } peers { permanent_uuid: "2ab1deb934114642a088159e52c51b56" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 37565 } }
I20260924 13:47:15.023613 15559 leader_election.cc:288] T b3495228ff2748f6afd56e75c1ad6ede P 2d405e4bdb8141389defffbc73190a5b [CANDIDATE]: Term 1 election: Requested vote from peers 21bfcfdb8dca41cca6a26000235952dd (127.13.149.193:40107), 2ab1deb934114642a088159e52c51b56 (127.13.149.195:37565)
I20260924 13:47:15.023855 15482 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b3495228ff2748f6afd56e75c1ad6ede" candidate_uuid: "2d405e4bdb8141389defffbc73190a5b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ab1deb934114642a088159e52c51b56"
I20260924 13:47:15.023936 15482 raft_consensus.cc:3037] T b3495228ff2748f6afd56e75c1ad6ede P 2ab1deb934114642a088159e52c51b56 [term 0 FOLLOWER]: Advancing to term 1
I20260924 13:47:15.023857 15196 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b3495228ff2748f6afd56e75c1ad6ede" candidate_uuid: "2d405e4bdb8141389defffbc73190a5b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "21bfcfdb8dca41cca6a26000235952dd"
I20260924 13:47:15.023967 15196 raft_consensus.cc:3037] T b3495228ff2748f6afd56e75c1ad6ede P 21bfcfdb8dca41cca6a26000235952dd [term 0 FOLLOWER]: Advancing to term 1
I20260924 13:47:15.024600 15196 raft_consensus.cc:2454] T b3495228ff2748f6afd56e75c1ad6ede P 21bfcfdb8dca41cca6a26000235952dd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2d405e4bdb8141389defffbc73190a5b in term 1.
I20260924 13:47:15.024602 15482 raft_consensus.cc:2454] T b3495228ff2748f6afd56e75c1ad6ede P 2ab1deb934114642a088159e52c51b56 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2d405e4bdb8141389defffbc73190a5b in term 1.
I20260924 13:47:15.024833 15270 leader_election.cc:302] T b3495228ff2748f6afd56e75c1ad6ede P 2d405e4bdb8141389defffbc73190a5b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ab1deb934114642a088159e52c51b56, 2d405e4bdb8141389defffbc73190a5b; no voters:
I20260924 13:47:15.024964 15559 raft_consensus.cc:2781] T b3495228ff2748f6afd56e75c1ad6ede P 2d405e4bdb8141389defffbc73190a5b [term 1 FOLLOWER]: Leader election won for term 1
I20260924 13:47:15.025031 15559 raft_consensus.cc:686] T b3495228ff2748f6afd56e75c1ad6ede P 2d405e4bdb8141389defffbc73190a5b [term 1 LEADER]: Becoming Leader. State: Replica: 2d405e4bdb8141389defffbc73190a5b, State: Running, Role: LEADER
I20260924 13:47:15.025081 15559 consensus_queue.cc:237] T b3495228ff2748f6afd56e75c1ad6ede P 2d405e4bdb8141389defffbc73190a5b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21bfcfdb8dca41cca6a26000235952dd" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 40107 } } peers { permanent_uuid: "2d405e4bdb8141389defffbc73190a5b" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 36491 } } peers { permanent_uuid: "2ab1deb934114642a088159e52c51b56" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 37565 } }
I20260924 13:47:15.025389 15581 tablet_replica.cc:439] T b3495228ff2748f6afd56e75c1ad6ede P 2d405e4bdb8141389defffbc73190a5b: TxnStatusTablet state changed. Reason: New leader 2d405e4bdb8141389defffbc73190a5b. Latest consensus state: current_term: 1 leader_uuid: "2d405e4bdb8141389defffbc73190a5b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21bfcfdb8dca41cca6a26000235952dd" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 40107 } } peers { permanent_uuid: "2d405e4bdb8141389defffbc73190a5b" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 36491 } } peers { permanent_uuid: "2ab1deb934114642a088159e52c51b56" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 37565 } } }
I20260924 13:47:15.025480 15581 tablet_replica.cc:442] T b3495228ff2748f6afd56e75c1ad6ede P 2d405e4bdb8141389defffbc73190a5b: This TxnStatusTablet replica's current role is: LEADER
I20260924 13:47:15.025702 15593 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260924 13:47:15.025839 15593 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260924 13:47:15.025753 15035 catalog_manager.cc:5495] T b3495228ff2748f6afd56e75c1ad6ede P 2d405e4bdb8141389defffbc73190a5b reported cstate change: term changed from 0 to 1, leader changed from <none> to 2d405e4bdb8141389defffbc73190a5b (127.13.149.194). New cstate: current_term: 1 leader_uuid: "2d405e4bdb8141389defffbc73190a5b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21bfcfdb8dca41cca6a26000235952dd" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 40107 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2d405e4bdb8141389defffbc73190a5b" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 36491 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2ab1deb934114642a088159e52c51b56" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 37565 } health_report { overall_health: UNKNOWN } } }
I20260924 13:47:15.051604 15547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:15.048507 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41774 (local address 127.13.149.194:36491)
0924 13:47:15.048556 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0924 13:47:15.048558 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:47:15.048569 (+ 11us) server_negotiation.cc:407] Connection header received
0924 13:47:15.048650 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:47:15.048651 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:47:15.048673 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:47:15.048737 (+ 64us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 13:47:15.048863 (+ 126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:15.049656 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:15.050434 (+ 778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:15.050763 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:15.050935 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:47:15.051119 (+ 184us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 13:47:15.051275 (+ 156us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 13:47:15.051349 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0924 13:47:15.051484 (+ 135us) server_negotiation.cc:299] Negotiation successful
0924 13:47:15.051512 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260924 13:47:15.051625 15588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:15.048426 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.194:36491 (local address 127.0.0.1:41774)
0924 13:47:15.048510 (+ 84us) negotiation.cc:107] Waiting for socket to connect
0924 13:47:15.048523 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0924 13:47:15.048577 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:47:15.048753 (+ 176us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:47:15.048754 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:47:15.048755 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0924 13:47:15.048806 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:47:15.048813 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:47:15.049750 (+ 937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:47:15.049751 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:47:15.050353 (+ 602us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:47:15.050355 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:47:15.050943 (+ 588us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:47:15.050944 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:47:15.051004 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:47:15.051025 (+ 21us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0924 13:47:15.051395 (+ 370us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0924 13:47:15.051408 (+ 13us) client_negotiation.cc:769] Sending connection context
0924 13:47:15.051480 (+ 72us) client_negotiation.cc:240] Negotiation successful
0924 13:47:15.051543 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42}
I20260924 13:47:15.052778 15196 raft_consensus.cc:1261] T b3495228ff2748f6afd56e75c1ad6ede P 21bfcfdb8dca41cca6a26000235952dd [term 1 FOLLOWER]: Refusing update from remote peer 2d405e4bdb8141389defffbc73190a5b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 13:47:15.052857 15482 raft_consensus.cc:1261] T b3495228ff2748f6afd56e75c1ad6ede P 2ab1deb934114642a088159e52c51b56 [term 1 FOLLOWER]: Refusing update from remote peer 2d405e4bdb8141389defffbc73190a5b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 13:47:15.053103 15581 consensus_queue.cc:1035] T b3495228ff2748f6afd56e75c1ad6ede P 2d405e4bdb8141389defffbc73190a5b [LEADER]: Connected to new peer: Peer: permanent_uuid: "21bfcfdb8dca41cca6a26000235952dd" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 40107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 13:47:15.053265 15559 consensus_queue.cc:1035] T b3495228ff2748f6afd56e75c1ad6ede P 2d405e4bdb8141389defffbc73190a5b [LEADER]: Connected to new peer: Peer: permanent_uuid: "2ab1deb934114642a088159e52c51b56" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 37565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 13:47:15.053867 15557 tablet_replica.cc:439] T b3495228ff2748f6afd56e75c1ad6ede P 21bfcfdb8dca41cca6a26000235952dd: TxnStatusTablet state changed. Reason: New leader 2d405e4bdb8141389defffbc73190a5b. Latest consensus state: current_term: 1 leader_uuid: "2d405e4bdb8141389defffbc73190a5b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21bfcfdb8dca41cca6a26000235952dd" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 40107 } } peers { permanent_uuid: "2d405e4bdb8141389defffbc73190a5b" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 36491 } } peers { permanent_uuid: "2ab1deb934114642a088159e52c51b56" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 37565 } } }
I20260924 13:47:15.053961 15557 tablet_replica.cc:442] T b3495228ff2748f6afd56e75c1ad6ede P 21bfcfdb8dca41cca6a26000235952dd: This TxnStatusTablet replica's current role is: FOLLOWER
I20260924 13:47:15.053972 15569 tablet_replica.cc:439] T b3495228ff2748f6afd56e75c1ad6ede P 2ab1deb934114642a088159e52c51b56: TxnStatusTablet state changed. Reason: New leader 2d405e4bdb8141389defffbc73190a5b. Latest consensus state: current_term: 1 leader_uuid: "2d405e4bdb8141389defffbc73190a5b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21bfcfdb8dca41cca6a26000235952dd" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 40107 } } peers { permanent_uuid: "2d405e4bdb8141389defffbc73190a5b" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 36491 } } peers { permanent_uuid: "2ab1deb934114642a088159e52c51b56" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 37565 } } }
I20260924 13:47:15.054057 15569 tablet_replica.cc:442] T b3495228ff2748f6afd56e75c1ad6ede P 2ab1deb934114642a088159e52c51b56: This TxnStatusTablet replica's current role is: FOLLOWER
I20260924 13:47:15.054859 15559 tablet_replica.cc:439] T b3495228ff2748f6afd56e75c1ad6ede P 2d405e4bdb8141389defffbc73190a5b: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "2d405e4bdb8141389defffbc73190a5b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21bfcfdb8dca41cca6a26000235952dd" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 40107 } } peers { permanent_uuid: "2d405e4bdb8141389defffbc73190a5b" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 36491 } } peers { permanent_uuid: "2ab1deb934114642a088159e52c51b56" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 37565 } } }
I20260924 13:47:15.054951 15559 tablet_replica.cc:442] T b3495228ff2748f6afd56e75c1ad6ede P 2d405e4bdb8141389defffbc73190a5b: This TxnStatusTablet replica's current role is: LEADER
I20260924 13:47:15.054986 15559 tablet_replica.cc:439] T b3495228ff2748f6afd56e75c1ad6ede P 2d405e4bdb8141389defffbc73190a5b: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "2d405e4bdb8141389defffbc73190a5b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21bfcfdb8dca41cca6a26000235952dd" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 40107 } } peers { permanent_uuid: "2d405e4bdb8141389defffbc73190a5b" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 36491 } } peers { permanent_uuid: "2ab1deb934114642a088159e52c51b56" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 37565 } } }
I20260924 13:47:15.055024 15559 tablet_replica.cc:442] T b3495228ff2748f6afd56e75c1ad6ede P 2d405e4bdb8141389defffbc73190a5b: This TxnStatusTablet replica's current role is: LEADER
I20260924 13:47:15.055769 15564 mvcc.cc:205] Tried to move back new op lower bound from 7332895273174327296 to 7332895273063239680. Current Snapshot: MvccSnapshot[applied={T|T < 7332895273174327296}]
I20260924 13:47:15.055745 15557 tablet_replica.cc:439] T b3495228ff2748f6afd56e75c1ad6ede P 21bfcfdb8dca41cca6a26000235952dd: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "2d405e4bdb8141389defffbc73190a5b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21bfcfdb8dca41cca6a26000235952dd" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 40107 } } peers { permanent_uuid: "2d405e4bdb8141389defffbc73190a5b" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 36491 } } peers { permanent_uuid: "2ab1deb934114642a088159e52c51b56" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 37565 } } }
I20260924 13:47:15.055835 15557 tablet_replica.cc:442] T b3495228ff2748f6afd56e75c1ad6ede P 21bfcfdb8dca41cca6a26000235952dd: This TxnStatusTablet replica's current role is: FOLLOWER
I20260924 13:47:15.055850 15569 tablet_replica.cc:439] T b3495228ff2748f6afd56e75c1ad6ede P 2ab1deb934114642a088159e52c51b56: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "2d405e4bdb8141389defffbc73190a5b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21bfcfdb8dca41cca6a26000235952dd" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 40107 } } peers { permanent_uuid: "2d405e4bdb8141389defffbc73190a5b" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 36491 } } peers { permanent_uuid: "2ab1deb934114642a088159e52c51b56" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 37565 } } }
I20260924 13:47:15.055927 15569 tablet_replica.cc:442] T b3495228ff2748f6afd56e75c1ad6ede P 2ab1deb934114642a088159e52c51b56: This TxnStatusTablet replica's current role is: FOLLOWER
I20260924 13:47:15.059916 15547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:15.057391 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41780 (local address 127.13.149.194:36491)
0924 13:47:15.057428 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0924 13:47:15.057430 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:47:15.057439 (+ 9us) server_negotiation.cc:407] Connection header received
0924 13:47:15.057506 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:47:15.057507 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:47:15.057516 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:47:15.057548 (+ 32us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 13:47:15.057674 (+ 126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:15.058144 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:15.059264 (+ 1120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:15.059475 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:15.059543 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:47:15.059697 (+ 154us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 13:47:15.059727 (+ 30us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 13:47:15.059762 (+ 35us) server_negotiation.cc:1035] Waiting for connection context
0924 13:47:15.059825 (+ 63us) server_negotiation.cc:299] Negotiation successful
0924 13:47:15.059849 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260924 13:47:15.065299 15547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:15.063001 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.149.194:53973 (local address 127.13.149.194:36491)
0924 13:47:15.063047 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0924 13:47:15.063049 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:47:15.063119 (+ 70us) server_negotiation.cc:407] Connection header received
0924 13:47:15.063216 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:47:15.063217 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:47:15.063227 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:47:15.063265 (+ 38us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 13:47:15.063411 (+ 146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:15.063868 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:15.064599 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:15.064773 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:15.064827 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:47:15.065041 (+ 214us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 13:47:15.065070 (+ 29us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 13:47:15.065123 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0924 13:47:15.065178 (+ 55us) server_negotiation.cc:299] Negotiation successful
0924 13:47:15.065202 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":29}
I20260924 13:47:15.065313 15601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:15.062849 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.194:36491 (local address 127.13.149.194:53973)
0924 13:47:15.063044 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0924 13:47:15.063071 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0924 13:47:15.063161 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:47:15.063273 (+ 112us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:47:15.063274 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:47:15.063276 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0924 13:47:15.063353 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:47:15.063355 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:47:15.063936 (+ 581us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:47:15.063937 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:47:15.064525 (+ 588us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:47:15.064527 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:47:15.064831 (+ 304us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:47:15.064832 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:47:15.064904 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:47:15.064963 (+ 59us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0924 13:47:15.065127 (+ 164us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0924 13:47:15.065128 (+ 1us) client_negotiation.cc:769] Sending connection context
0924 13:47:15.065174 (+ 46us) client_negotiation.cc:240] Negotiation successful
0924 13:47:15.065224 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"mutex_wait_us":2,"thread_start_us":92,"threads_started":1}
I20260924 13:47:15.068156 15482 raft_consensus.cc:1261] T c9909c5b7d014e389b30787c03d5dffd P 2ab1deb934114642a088159e52c51b56 [term 1 FOLLOWER]: Refusing update from remote peer 2d405e4bdb8141389defffbc73190a5b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 13:47:15.068159 15196 raft_consensus.cc:1261] T c9909c5b7d014e389b30787c03d5dffd P 21bfcfdb8dca41cca6a26000235952dd [term 1 FOLLOWER]: Refusing update from remote peer 2d405e4bdb8141389defffbc73190a5b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 13:47:15.068424 15598 consensus_queue.cc:1035] T c9909c5b7d014e389b30787c03d5dffd P 2d405e4bdb8141389defffbc73190a5b [LEADER]: Connected to new peer: Peer: permanent_uuid: "21bfcfdb8dca41cca6a26000235952dd" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 40107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 13:47:15.068575 15581 consensus_queue.cc:1035] T c9909c5b7d014e389b30787c03d5dffd P 2d405e4bdb8141389defffbc73190a5b [LEADER]: Connected to new peer: Peer: permanent_uuid: "2ab1deb934114642a088159e52c51b56" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 37565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 13:47:15.085045 15037 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44722:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 13:47:15.085173 15037 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44722:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 13:47:15.085948 15037 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 244154f1876f4f849fa9af203bd8fe2c: Sending DeleteTablet for 3 replicas of tablet c9909c5b7d014e389b30787c03d5dffd
I20260924 13:47:15.086385 15317 tablet_service.cc:1514] Processing DeleteTablet for tablet c9909c5b7d014e389b30787c03d5dffd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 13:47:15 UTC) from {username='kudu', principal='kudu/127.13.149.254@KRBTEST.COM'} at 127.0.0.1:41768
I20260924 13:47:15.086450 15462 tablet_service.cc:1514] Processing DeleteTablet for tablet c9909c5b7d014e389b30787c03d5dffd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 13:47:15 UTC) from {username='kudu', principal='kudu/127.13.149.254@KRBTEST.COM'} at 127.0.0.1:47654
I20260924 13:47:15.086505 15575 tablet_replica.cc:330] T c9909c5b7d014e389b30787c03d5dffd P 2d405e4bdb8141389defffbc73190a5b: stopping tablet replica
I20260924 13:47:15.086563 15575 raft_consensus.cc:2229] T c9909c5b7d014e389b30787c03d5dffd P 2d405e4bdb8141389defffbc73190a5b [term 1 LEADER]: Raft consensus shutting down.
I20260924 13:47:15.086567 15576 tablet_replica.cc:330] T c9909c5b7d014e389b30787c03d5dffd P 2ab1deb934114642a088159e52c51b56: stopping tablet replica
I20260924 13:47:15.086628 15576 raft_consensus.cc:2229] T c9909c5b7d014e389b30787c03d5dffd P 2ab1deb934114642a088159e52c51b56 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 13:47:15.086668 15576 raft_consensus.cc:2258] T c9909c5b7d014e389b30787c03d5dffd P 2ab1deb934114642a088159e52c51b56 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 13:47:15.086663 15575 raft_consensus.cc:2258] T c9909c5b7d014e389b30787c03d5dffd P 2d405e4bdb8141389defffbc73190a5b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 13:47:15.086954 15176 tablet_service.cc:1514] Processing DeleteTablet for tablet c9909c5b7d014e389b30787c03d5dffd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 13:47:15 UTC) from {username='kudu', principal='kudu/127.13.149.254@KRBTEST.COM'} at 127.0.0.1:38352
I20260924 13:47:15.087008 15576 ts_tablet_manager.cc:1905] T c9909c5b7d014e389b30787c03d5dffd P 2ab1deb934114642a088159e52c51b56: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 13:47:15.087071 15574 tablet_replica.cc:330] T c9909c5b7d014e389b30787c03d5dffd P 21bfcfdb8dca41cca6a26000235952dd: stopping tablet replica
I20260924 13:47:15.087107 15575 ts_tablet_manager.cc:1905] T c9909c5b7d014e389b30787c03d5dffd P 2d405e4bdb8141389defffbc73190a5b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 13:47:15.087126 15574 raft_consensus.cc:2229] T c9909c5b7d014e389b30787c03d5dffd P 21bfcfdb8dca41cca6a26000235952dd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 13:47:15.087167 15574 raft_consensus.cc:2258] T c9909c5b7d014e389b30787c03d5dffd P 21bfcfdb8dca41cca6a26000235952dd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 13:47:15.087764 15574 ts_tablet_manager.cc:1905] T c9909c5b7d014e389b30787c03d5dffd P 21bfcfdb8dca41cca6a26000235952dd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 13:47:15.088860 15576 ts_tablet_manager.cc:1918] T c9909c5b7d014e389b30787c03d5dffd P 2ab1deb934114642a088159e52c51b56: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260924 13:47:15.088867 15575 ts_tablet_manager.cc:1918] T c9909c5b7d014e389b30787c03d5dffd P 2d405e4bdb8141389defffbc73190a5b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260924 13:47:15.088937 15575 log.cc:1196] T c9909c5b7d014e389b30787c03d5dffd P 2d405e4bdb8141389defffbc73190a5b: Deleting WAL directory at /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790257629041645-13911-0/minicluster-data/ts-1/wal/wals/c9909c5b7d014e389b30787c03d5dffd
I20260924 13:47:15.088951 15576 log.cc:1196] T c9909c5b7d014e389b30787c03d5dffd P 2ab1deb934114642a088159e52c51b56: Deleting WAL directory at /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790257629041645-13911-0/minicluster-data/ts-2/wal/wals/c9909c5b7d014e389b30787c03d5dffd
I20260924 13:47:15.089159 15576 ts_tablet_manager.cc:1939] T c9909c5b7d014e389b30787c03d5dffd P 2ab1deb934114642a088159e52c51b56: Deleting consensus metadata
I20260924 13:47:15.089159 15575 ts_tablet_manager.cc:1939] T c9909c5b7d014e389b30787c03d5dffd P 2d405e4bdb8141389defffbc73190a5b: Deleting consensus metadata
I20260924 13:47:15.089588 15574 ts_tablet_manager.cc:1918] T c9909c5b7d014e389b30787c03d5dffd P 21bfcfdb8dca41cca6a26000235952dd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260924 13:47:15.089648 15574 log.cc:1196] T c9909c5b7d014e389b30787c03d5dffd P 21bfcfdb8dca41cca6a26000235952dd: Deleting WAL directory at /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790257629041645-13911-0/minicluster-data/ts-0/wal/wals/c9909c5b7d014e389b30787c03d5dffd
I20260924 13:47:15.089840 15574 ts_tablet_manager.cc:1939] T c9909c5b7d014e389b30787c03d5dffd P 21bfcfdb8dca41cca6a26000235952dd: Deleting consensus metadata
I20260924 13:47:15.090008 15024 catalog_manager.cc:4841] TS 2d405e4bdb8141389defffbc73190a5b (127.13.149.194:36491): tablet c9909c5b7d014e389b30787c03d5dffd (table test-table [id=53908ce3aaa34a4892350cb4e13f76c5]) successfully deleted
I20260924 13:47:15.090090 15024 catalog_manager.cc:4841] TS 2ab1deb934114642a088159e52c51b56 (127.13.149.195:37565): tablet c9909c5b7d014e389b30787c03d5dffd (table test-table [id=53908ce3aaa34a4892350cb4e13f76c5]) successfully deleted
I20260924 13:47:15.090392 15024 catalog_manager.cc:4841] TS 21bfcfdb8dca41cca6a26000235952dd (127.13.149.193:40107): tablet c9909c5b7d014e389b30787c03d5dffd (table test-table [id=53908ce3aaa34a4892350cb4e13f76c5]) successfully deleted
Sep 24 13:47:15 dist-test-slave-sjl2 krb5kdc[14992](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257634, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.13.149.193@KRBTEST.COM
I20260924 13:47:15.098874 15550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:15.089048 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38368 (local address 127.13.149.193:40107)
0924 13:47:15.089093 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0924 13:47:15.089096 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 13:47:15.089466 (+ 370us) server_negotiation.cc:407] Connection header received
0924 13:47:15.089541 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:47:15.089542 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:47:15.089558 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:47:15.089602 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:47:15.090450 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:15.090979 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:15.092045 (+ 1066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:15.092236 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:15.092311 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:47:15.095837 (+ 3526us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:47:15.095850 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:47:15.095850 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 13:47:15.095859 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:47:15.097643 (+ 1784us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:47:15.098069 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:47:15.098072 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:47:15.098072 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:47:15.098138 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:47:15.098338 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:47:15.098338 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:47:15.098338 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:47:15.098475 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:47:15.098569 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0924 13:47:15.098709 (+ 140us) server_negotiation.cc:299] Negotiation successful
0924 13:47:15.098778 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
W20260924 13:47:15.099177 15236 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38368
I20260924 13:47:15.105818 15088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:15.100118 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44754 (local address 127.13.149.254:35189)
0924 13:47:15.100160 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0924 13:47:15.100162 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:47:15.100255 (+ 93us) server_negotiation.cc:407] Connection header received
0924 13:47:15.100323 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:47:15.100324 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:47:15.100334 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:47:15.100378 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:47:15.100852 (+ 474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:15.101470 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:15.102140 (+ 670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:15.102348 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:15.102402 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:47:15.103533 (+ 1131us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:47:15.103535 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:47:15.103535 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 13:47:15.103542 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:47:15.104799 (+ 1257us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:47:15.105131 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:47:15.105132 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:47:15.105133 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:47:15.105188 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:47:15.105349 (+ 161us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:47:15.105349 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:47:15.105350 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:47:15.105464 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:47:15.105553 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0924 13:47:15.105646 (+ 93us) server_negotiation.cc:299] Negotiation successful
0924 13:47:15.105724 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
W20260924 13:47:15.106142 15037 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44754
I20260924 13:47:15.106812 13911 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJN9WRF/build/release/bin/kudu with pid 15110
I20260924 13:47:15.111847 13911 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJN9WRF/build/release/bin/kudu with pid 15251
I20260924 13:47:15.117291 13911 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJN9WRF/build/release/bin/kudu with pid 15396
I20260924 13:47:15.122423 13911 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJN9WRF/build/release/bin/kudu with pid 15005
2026-09-24T13:47:15Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (2396 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790257629041645-13911-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 13:47:15 dist-test-slave-sjl2 krb5kdc[15612](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 13:47:15 dist-test-slave-sjl2 krb5kdc[15612](info): set up 2 sockets
Sep 24 13:47:15 dist-test-slave-sjl2 krb5kdc[15612](info): commencing operation
krb5kdc: starting...
W20260924 13:47:16.085598 13911 mini_kdc.cc:121] Time spent starting KDC: real 0.942s user 0.002s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790257629041645-13911-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790257629041645-13911-0/krb5kdc/test-user.keytab.
Sep 24 13:47:16 dist-test-slave-sjl2 krb5kdc[15612](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257636, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T13:47:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T13:47:16Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.149.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.149.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.149.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.254.keytab.
Entry for principal kudu/127.13.149.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.254.keytab.
I20260924 13:47:16.208292 13911 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJN9WRF/build/release/bin/kudu
/tmp/dist-test-taskJN9WRF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790257629041645-13911-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790257629041645-13911-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790257629041645-13911-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790257629041645-13911-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.149.254:40663
--webserver_interface=127.13.149.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.254.keytab
--principal=kudu/127.13.149.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.149.212:42859
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.149.254:40663
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790257629041645-13911-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790257629041645-13911-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790257629041645-13911-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 13:47:16.290915 15625 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 13:47:16.291146 15625 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 13:47:16.291172 15625 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 13:47:16.292361 15625 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 13:47:16.292402 15625 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260924 13:47:16.292413 15625 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 13:47:16.292425 15625 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 13:47:16.292436 15625 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 13:47:16.292446 15625 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 13:47:16.293977 15625 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.149.212:42859
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790257629041645-13911-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790257629041645-13911-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.149.254:40663
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.254.keytab
--principal=kudu/127.13.149.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.149.254:40663
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790257629041645-13911-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.149.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15625
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790257629041645-13911-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 12:29:37 UTC on ccycloud.quasar-iqzbxn.root.comops.site
build id 4670106
I20260924 13:47:16.294202 15625 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 13:47:16.294390 15625 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 13:47:16.299233 15633 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 13:47:16.299268 15631 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 13:47:16.299268 15630 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 13:47:16.299495 15625 server_base.cc:1034] running on GCE node
I20260924 13:47:16.299758 15625 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 13:47:16.300117 15625 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 13:47:16.301273 15625 hybrid_clock.cc:648] HybridClock initialized: now 1790257636301256 us; error 31 us; skew 500 ppm
Sep 24 13:47:16 dist-test-slave-sjl2 krb5kdc[15612](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257636, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.149.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 13:47:16.304020 15625 init.cc:377] Logged in from keytab as kudu/127.13.149.254@KRBTEST.COM (short username kudu)
I20260924 13:47:16.304508 15625 webserver.cc:460] Webserver started at http://127.13.149.254:41695/ using document root <none> and password file <none>
I20260924 13:47:16.304746 15625 fs_manager.cc:359] Metadata directory not provided
I20260924 13:47:16.304795 15625 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 13:47:16.304946 15625 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 13:47:16.306049 15625 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790257629041645-13911-0/minicluster-data/master-0/data/instance:
uuid: "4dbf8e8088ce4e99ba854b50554eeb6a"
format_stamp: "Formatted at 2026-09-24 13:47:16 on dist-test-slave-sjl2"
server_key: "86ef5df590d3b4d566c29607de71825d"
server_key_iv: "2c21e06fd0c617bfdccd2a99f09c1ab7"
server_key_version: "encryptionkey@0"
I20260924 13:47:16.306394 15625 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790257629041645-13911-0/minicluster-data/master-0/wal/instance:
uuid: "4dbf8e8088ce4e99ba854b50554eeb6a"
format_stamp: "Formatted at 2026-09-24 13:47:16 on dist-test-slave-sjl2"
server_key: "86ef5df590d3b4d566c29607de71825d"
server_key_iv: "2c21e06fd0c617bfdccd2a99f09c1ab7"
server_key_version: "encryptionkey@0"
I20260924 13:47:16.307770 15625 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260924 13:47:16.308580 15641 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 13:47:16.308780 15625 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260924 13:47:16.308873 15625 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790257629041645-13911-0/minicluster-data/master-0/data,/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790257629041645-13911-0/minicluster-data/master-0/wal
uuid: "4dbf8e8088ce4e99ba854b50554eeb6a"
format_stamp: "Formatted at 2026-09-24 13:47:16 on dist-test-slave-sjl2"
server_key: "86ef5df590d3b4d566c29607de71825d"
server_key_iv: "2c21e06fd0c617bfdccd2a99f09c1ab7"
server_key_version: "encryptionkey@0"
I20260924 13:47:16.308997 15625 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790257629041645-13911-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790257629041645-13911-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790257629041645-13911-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 13:47:16.327046 15625 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 13:47:16.329984 15625 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 13:47:16.330274 15625 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 13:47:16.335784 15625 rpc_server.cc:307] RPC server started. Bound to: 127.13.149.254:40663
I20260924 13:47:16.335844 15704 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.149.254:40663 every 8 connection(s)
I20260924 13:47:16.336210 15625 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790257629041645-13911-0/minicluster-data/master-0/data/info.pb
I20260924 13:47:16.336969 15705 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 13:47:16.339468 15705 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4dbf8e8088ce4e99ba854b50554eeb6a: Bootstrap starting.
I20260924 13:47:16.340099 15705 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4dbf8e8088ce4e99ba854b50554eeb6a: Neither blocks nor log segments found. Creating new log.
I20260924 13:47:16.340381 15705 log.cc:824] T 00000000000000000000000000000000 P 4dbf8e8088ce4e99ba854b50554eeb6a: Log is configured to *not* fsync() on all Append() calls
I20260924 13:47:16.341180 15705 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4dbf8e8088ce4e99ba854b50554eeb6a: No bootstrap required, opened a new log
I20260924 13:47:16.343397 15705 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4dbf8e8088ce4e99ba854b50554eeb6a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4dbf8e8088ce4e99ba854b50554eeb6a" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 40663 } }
I20260924 13:47:16.343613 13911 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJN9WRF/build/release/bin/kudu as pid 15625
I20260924 13:47:16.343626 15705 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4dbf8e8088ce4e99ba854b50554eeb6a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 13:47:16.343662 15705 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4dbf8e8088ce4e99ba854b50554eeb6a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4dbf8e8088ce4e99ba854b50554eeb6a, State: Initialized, Role: FOLLOWER
I20260924 13:47:16.343689 13911 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790257629041645-13911-0/minicluster-data/master-0/wal/instance
I20260924 13:47:16.343873 13911 external_mini_cluster.cc:1428] Setting key acc577dfbaf99eff4ce8bc2df45ba877
I20260924 13:47:16.343832 15705 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4dbf8e8088ce4e99ba854b50554eeb6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4dbf8e8088ce4e99ba854b50554eeb6a" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 40663 } }
I20260924 13:47:16.343916 15705 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4dbf8e8088ce4e99ba854b50554eeb6a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 13:47:16.343948 15705 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4dbf8e8088ce4e99ba854b50554eeb6a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 13:47:16.344002 15705 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4dbf8e8088ce4e99ba854b50554eeb6a [term 0 FOLLOWER]: Advancing to term 1
I20260924 13:47:16.344758 15705 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4dbf8e8088ce4e99ba854b50554eeb6a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4dbf8e8088ce4e99ba854b50554eeb6a" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 40663 } }
I20260924 13:47:16.344861 15705 leader_election.cc:302] T 00000000000000000000000000000000 P 4dbf8e8088ce4e99ba854b50554eeb6a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4dbf8e8088ce4e99ba854b50554eeb6a; no voters:
I20260924 13:47:16.345047 15705 leader_election.cc:288] T 00000000000000000000000000000000 P 4dbf8e8088ce4e99ba854b50554eeb6a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 13:47:16.345083 15710 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4dbf8e8088ce4e99ba854b50554eeb6a [term 1 FOLLOWER]: Leader election won for term 1
I20260924 13:47:16.345189 15710 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4dbf8e8088ce4e99ba854b50554eeb6a [term 1 LEADER]: Becoming Leader. State: Replica: 4dbf8e8088ce4e99ba854b50554eeb6a, State: Running, Role: LEADER
I20260924 13:47:16.345274 15710 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4dbf8e8088ce4e99ba854b50554eeb6a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4dbf8e8088ce4e99ba854b50554eeb6a" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 40663 } }
I20260924 13:47:16.345325 15705 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4dbf8e8088ce4e99ba854b50554eeb6a [sys.catalog]: configured and running, proceeding with master startup.
I20260924 13:47:16.345639 15711 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4dbf8e8088ce4e99ba854b50554eeb6a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4dbf8e8088ce4e99ba854b50554eeb6a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4dbf8e8088ce4e99ba854b50554eeb6a" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 40663 } } }
I20260924 13:47:16.345716 15711 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4dbf8e8088ce4e99ba854b50554eeb6a [sys.catalog]: This master's current role is: LEADER
I20260924 13:47:16.345862 15712 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4dbf8e8088ce4e99ba854b50554eeb6a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4dbf8e8088ce4e99ba854b50554eeb6a. Latest consensus state: current_term: 1 leader_uuid: "4dbf8e8088ce4e99ba854b50554eeb6a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4dbf8e8088ce4e99ba854b50554eeb6a" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 40663 } } }
I20260924 13:47:16.345940 15712 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4dbf8e8088ce4e99ba854b50554eeb6a [sys.catalog]: This master's current role is: LEADER
I20260924 13:47:16.346375 15715 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 13:47:16.346916 15715 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 13:47:16.348783 15715 catalog_manager.cc:1347] Generated new cluster ID: bf3eefaf0bae428f9080aa3dd6240f76
I20260924 13:47:16.348857 15715 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 24 13:47:16 dist-test-slave-sjl2 krb5kdc[15612](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257636, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.149.254@KRBTEST.COM
I20260924 13:47:16.358788 15709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:16.344450 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42850 (local address 127.13.149.254:40663)
0924 13:47:16.344631 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0924 13:47:16.344640 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0924 13:47:16.344668 (+ 28us) server_negotiation.cc:407] Connection header received
0924 13:47:16.344776 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:47:16.344783 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:47:16.344847 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:47:16.345052 (+ 205us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:47:16.346348 (+ 1296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:16.347585 (+ 1237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:16.348557 (+ 972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:16.349226 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:16.349502 (+ 276us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:47:16.353428 (+ 3926us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:47:16.353437 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:47:16.353438 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 13:47:16.353461 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:47:16.355875 (+ 2414us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:47:16.356654 (+ 779us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:47:16.356659 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:47:16.356660 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:47:16.356830 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:47:16.357061 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:47:16.357062 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:47:16.357062 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:47:16.357479 (+ 417us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:47:16.357649 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0924 13:47:16.358310 (+ 661us) server_negotiation.cc:299] Negotiation successful
0924 13:47:16.358490 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":129,"thread_start_us":84,"threads_started":1}
I20260924 13:47:16.363775 15715 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 13:47:16.364372 15715 catalog_manager.cc:1504] Loading token signing keys...
I20260924 13:47:16.370893 15715 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4dbf8e8088ce4e99ba854b50554eeb6a: Generated new TSK 0
I20260924 13:47:16.371102 15715 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.13.149.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.149.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.149.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.193.keytab.
Entry for principal kudu/127.13.149.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.193.keytab.
I20260924 13:47:16.397899 13911 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJN9WRF/build/release/bin/kudu
/tmp/dist-test-taskJN9WRF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790257629041645-13911-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790257629041645-13911-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790257629041645-13911-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790257629041645-13911-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.149.193:0
--local_ip_for_outbound_sockets=127.13.149.193
--webserver_interface=127.13.149.193
--webserver_port=0
--tserver_master_addrs=127.13.149.254:40663
--keytab_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.193.keytab
--principal=kudu/127.13.149.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.149.212:42859
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790257629041645-13911-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790257629041645-13911-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790257629041645-13911-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 13:47:16.483518 15731 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 13:47:16.483757 15731 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 13:47:16.483784 15731 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 13:47:16.485117 15731 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 13:47:16.485175 15731 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 13:47:16.485203 15731 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260924 13:47:16.485214 15731 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260924 13:47:16.485234 15731 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.149.193
I20260924 13:47:16.486871 15731 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.149.212:42859
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790257629041645-13911-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790257629041645-13911-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.193.keytab
--principal=kudu/127.13.149.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.149.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790257629041645-13911-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.149.193
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.13.149.254:40663
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15731
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.149.193
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790257629041645-13911-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 12:29:37 UTC on ccycloud.quasar-iqzbxn.root.comops.site
build id 4670106
I20260924 13:47:16.487129 15731 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 13:47:16.487375 15731 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 13:47:16.492460 15736 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 13:47:16.492372 15739 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 13:47:16.492399 15731 server_base.cc:1034] running on GCE node
W20260924 13:47:16.492378 15737 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 13:47:16.492785 15731 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 13:47:16.493078 15731 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 13:47:16.494236 15731 hybrid_clock.cc:648] HybridClock initialized: now 1790257636494223 us; error 32 us; skew 500 ppm
Sep 24 13:47:16 dist-test-slave-sjl2 krb5kdc[15612](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257636, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.149.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 13:47:16.497069 15731 init.cc:377] Logged in from keytab as kudu/127.13.149.193@KRBTEST.COM (short username kudu)
I20260924 13:47:16.497573 15731 webserver.cc:460] Webserver started at http://127.13.149.193:33953/ using document root <none> and password file <none>
I20260924 13:47:16.497763 15731 fs_manager.cc:359] Metadata directory not provided
I20260924 13:47:16.497807 15731 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 13:47:16.497902 15731 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 13:47:16.498970 15731 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790257629041645-13911-0/minicluster-data/ts-0/data/instance:
uuid: "58c281726a7d46c2b6c754c8a4fe5bf4"
format_stamp: "Formatted at 2026-09-24 13:47:16 on dist-test-slave-sjl2"
server_key: "e17e8fced0c8299cf6c6472606d354e6"
server_key_iv: "f8fd039eaa7473141b7c90b0fc9292d3"
server_key_version: "encryptionkey@0"
I20260924 13:47:16.499281 15731 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790257629041645-13911-0/minicluster-data/ts-0/wal/instance:
uuid: "58c281726a7d46c2b6c754c8a4fe5bf4"
format_stamp: "Formatted at 2026-09-24 13:47:16 on dist-test-slave-sjl2"
server_key: "e17e8fced0c8299cf6c6472606d354e6"
server_key_iv: "f8fd039eaa7473141b7c90b0fc9292d3"
server_key_version: "encryptionkey@0"
I20260924 13:47:16.500551 15731 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260924 13:47:16.501420 15746 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 13:47:16.501628 15731 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260924 13:47:16.501720 15731 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790257629041645-13911-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790257629041645-13911-0/minicluster-data/ts-0/wal
uuid: "58c281726a7d46c2b6c754c8a4fe5bf4"
format_stamp: "Formatted at 2026-09-24 13:47:16 on dist-test-slave-sjl2"
server_key: "e17e8fced0c8299cf6c6472606d354e6"
server_key_iv: "f8fd039eaa7473141b7c90b0fc9292d3"
server_key_version: "encryptionkey@0"
I20260924 13:47:16.501801 15731 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790257629041645-13911-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790257629041645-13911-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790257629041645-13911-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 13:47:16.513881 15731 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 13:47:16.516093 15731 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 13:47:16.516239 15731 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 13:47:16.516825 15731 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 13:47:16.516867 15731 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 13:47:16.516891 15731 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 13:47:16.516902 15731 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 24 13:47:16 dist-test-slave-sjl2 krb5kdc[15612](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257636, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.149.193@KRBTEST.COM for kudu/127.13.149.254@KRBTEST.COM
I20260924 13:47:16.524353 15731 rpc_server.cc:307] RPC server started. Bound to: 127.13.149.193:40235
I20260924 13:47:16.524451 15862 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.149.193:40235 every 8 connection(s)
I20260924 13:47:16.524776 15731 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790257629041645-13911-0/minicluster-data/ts-0/data/info.pb
I20260924 13:47:16.526919 15709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:16.517348 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.149.193:53417 (local address 127.13.149.254:40663)
0924 13:47:16.517420 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0924 13:47:16.517423 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 13:47:16.517655 (+ 232us) server_negotiation.cc:407] Connection header received
0924 13:47:16.518362 (+ 707us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:47:16.518363 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:47:16.518373 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:47:16.518426 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:47:16.519429 (+ 1003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:16.520032 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:16.521182 (+ 1150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:16.521409 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:16.521471 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:47:16.524114 (+ 2643us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:47:16.524116 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:47:16.524117 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 13:47:16.524127 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:47:16.525708 (+ 1581us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:47:16.526132 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:47:16.526133 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:47:16.526133 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:47:16.526192 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:47:16.526402 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:47:16.526403 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:47:16.526403 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:47:16.526514 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:47:16.526561 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0924 13:47:16.526782 (+ 221us) server_negotiation.cc:299] Negotiation successful
0924 13:47:16.526851 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260924 13:47:16.527114 15755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:16.517326 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:40663 (local address 127.13.149.193:53417)
0924 13:47:16.517506 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0924 13:47:16.517562 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0924 13:47:16.518235 (+ 673us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:47:16.518521 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:47:16.518525 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:47:16.518940 (+ 415us) client_negotiation.cc:189] Negotiated authn=SASL
0924 13:47:16.519311 (+ 371us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:47:16.519318 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:47:16.520143 (+ 825us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:47:16.520144 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:47:16.521088 (+ 944us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:47:16.521092 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:47:16.521514 (+ 422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:47:16.521515 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:47:16.521641 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:47:16.522171 (+ 530us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 13:47:16.522180 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0924 13:47:16.523976 (+ 1796us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 13:47:16.525814 (+ 1838us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 13:47:16.525821 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 13:47:16.525823 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 13:47:16.526060 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 13:47:16.526244 (+ 184us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 13:47:16.526244 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 13:47:16.526244 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 13:47:16.526355 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 13:47:16.526551 (+ 196us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 13:47:16.526554 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 13:47:16.526678 (+ 124us) client_negotiation.cc:769] Sending connection context
0924 13:47:16.526758 (+ 80us) client_negotiation.cc:240] Negotiation successful
0924 13:47:16.526895 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":91,"threads_started":1}
I20260924 13:47:16.527601 15863 heartbeater.cc:347] Connected to a master server at 127.13.149.254:40663
I20260924 13:47:16.527689 15863 heartbeater.cc:464] Registering TS with master...
I20260924 13:47:16.527899 15863 heartbeater.cc:511] Master 127.13.149.254:40663 requested a full tablet report, sending...
I20260924 13:47:16.528518 15657 ts_manager.cc:194] Registered new tserver with Master: 58c281726a7d46c2b6c754c8a4fe5bf4 (127.13.149.193:40235)
I20260924 13:47:16.529440 15657 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.149.193@KRBTEST.COM'} at 127.13.149.193:53417
I20260924 13:47:16.533061 13911 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJN9WRF/build/release/bin/kudu as pid 15731
I20260924 13:47:16.533149 13911 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790257629041645-13911-0/minicluster-data/ts-0/wal/instance
I20260924 13:47:16.533330 13911 external_mini_cluster.cc:1428] Setting key cb54a5e4fae203b6dcec6d0c2cf97ecc
I20260924 13:47:16.534614 15709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:16.528840 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.149.193:44739 (local address 127.13.149.254:40663)
0924 13:47:16.528886 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0924 13:47:16.528888 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:47:16.528964 (+ 76us) server_negotiation.cc:407] Connection header received
0924 13:47:16.529026 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:47:16.529026 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:47:16.529034 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:47:16.529074 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:47:16.529427 (+ 353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:16.529980 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:16.530726 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:16.530934 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:16.530992 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:47:16.531938 (+ 946us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:47:16.531940 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:47:16.531940 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 13:47:16.531946 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:47:16.533117 (+ 1171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:47:16.533579 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:47:16.533580 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:47:16.533580 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:47:16.533638 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:47:16.533846 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:47:16.533846 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:47:16.533847 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:47:16.533975 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:47:16.534078 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0924 13:47:16.534450 (+ 372us) server_negotiation.cc:299] Negotiation successful
0924 13:47:16.534546 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260924 13:47:16.534780 15869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:16.528749 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:40663 (local address 127.13.149.193:44739)
0924 13:47:16.528884 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0924 13:47:16.528895 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0924 13:47:16.528979 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:47:16.529086 (+ 107us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:47:16.529087 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:47:16.529292 (+ 205us) client_negotiation.cc:189] Negotiated authn=SASL
0924 13:47:16.529367 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:47:16.529369 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:47:16.530066 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:47:16.530067 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:47:16.530635 (+ 568us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:47:16.530638 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:47:16.530997 (+ 359us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:47:16.530998 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:47:16.531052 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:47:16.531511 (+ 459us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 13:47:16.531517 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0924 13:47:16.531847 (+ 330us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 13:47:16.533222 (+ 1375us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 13:47:16.533223 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 13:47:16.533223 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 13:47:16.533426 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 13:47:16.533707 (+ 281us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 13:47:16.533707 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 13:47:16.533708 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 13:47:16.533774 (+ 66us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 13:47:16.534180 (+ 406us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 13:47:16.534182 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 13:47:16.534290 (+ 108us) client_negotiation.cc:769] Sending connection context
0924 13:47:16.534416 (+ 126us) client_negotiation.cc:240] Negotiation successful
0924 13:47:16.534674 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":69,"threads_started":1}
WARNING: no policy specified for kudu/127.13.149.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.149.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.149.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.194.keytab.
Entry for principal kudu/127.13.149.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.194.keytab.
I20260924 13:47:16.558001 13911 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJN9WRF/build/release/bin/kudu
/tmp/dist-test-taskJN9WRF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790257629041645-13911-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790257629041645-13911-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790257629041645-13911-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790257629041645-13911-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.149.194:0
--local_ip_for_outbound_sockets=127.13.149.194
--webserver_interface=127.13.149.194
--webserver_port=0
--tserver_master_addrs=127.13.149.254:40663
--keytab_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.194.keytab
--principal=kudu/127.13.149.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.149.212:42859
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790257629041645-13911-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790257629041645-13911-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790257629041645-13911-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 13:47:16.640033 15872 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 13:47:16.640271 15872 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 13:47:16.640300 15872 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 13:47:16.641556 15872 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 13:47:16.641604 15872 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 13:47:16.641625 15872 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260924 13:47:16.641636 15872 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260924 13:47:16.641654 15872 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.149.194
I20260924 13:47:16.643178 15872 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.149.212:42859
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790257629041645-13911-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790257629041645-13911-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.194.keytab
--principal=kudu/127.13.149.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.149.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790257629041645-13911-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.149.194
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.13.149.254:40663
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15872
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.149.194
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790257629041645-13911-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 12:29:37 UTC on ccycloud.quasar-iqzbxn.root.comops.site
build id 4670106
I20260924 13:47:16.643414 15872 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 13:47:16.643615 15872 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 13:47:16.648950 15878 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 13:47:16.648967 15877 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 13:47:16.648900 15880 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 13:47:16.648970 15872 server_base.cc:1034] running on GCE node
I20260924 13:47:16.649334 15872 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 13:47:16.649580 15872 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 13:47:16.650751 15872 hybrid_clock.cc:648] HybridClock initialized: now 1790257636650727 us; error 43 us; skew 500 ppm
Sep 24 13:47:16 dist-test-slave-sjl2 krb5kdc[15612](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257636, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.149.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 13:47:16.653304 15872 init.cc:377] Logged in from keytab as kudu/127.13.149.194@KRBTEST.COM (short username kudu)
I20260924 13:47:16.653790 15872 webserver.cc:460] Webserver started at http://127.13.149.194:44063/ using document root <none> and password file <none>
I20260924 13:47:16.653978 15872 fs_manager.cc:359] Metadata directory not provided
I20260924 13:47:16.654019 15872 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 13:47:16.654110 15872 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 13:47:16.655138 15872 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790257629041645-13911-0/minicluster-data/ts-1/data/instance:
uuid: "19e79fbaeb2f4d8f9e4cb95bc73df894"
format_stamp: "Formatted at 2026-09-24 13:47:16 on dist-test-slave-sjl2"
server_key: "2d5652a77e1ee9a9149ab0206646e276"
server_key_iv: "0659c0d32d18da5884e5acac2d204319"
server_key_version: "encryptionkey@0"
I20260924 13:47:16.655452 15872 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790257629041645-13911-0/minicluster-data/ts-1/wal/instance:
uuid: "19e79fbaeb2f4d8f9e4cb95bc73df894"
format_stamp: "Formatted at 2026-09-24 13:47:16 on dist-test-slave-sjl2"
server_key: "2d5652a77e1ee9a9149ab0206646e276"
server_key_iv: "0659c0d32d18da5884e5acac2d204319"
server_key_version: "encryptionkey@0"
I20260924 13:47:16.656725 15872 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260924 13:47:16.657497 15887 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 13:47:16.657651 15872 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260924 13:47:16.657734 15872 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790257629041645-13911-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790257629041645-13911-0/minicluster-data/ts-1/wal
uuid: "19e79fbaeb2f4d8f9e4cb95bc73df894"
format_stamp: "Formatted at 2026-09-24 13:47:16 on dist-test-slave-sjl2"
server_key: "2d5652a77e1ee9a9149ab0206646e276"
server_key_iv: "0659c0d32d18da5884e5acac2d204319"
server_key_version: "encryptionkey@0"
I20260924 13:47:16.657809 15872 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790257629041645-13911-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790257629041645-13911-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790257629041645-13911-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 13:47:16.668481 15872 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 13:47:16.670676 15872 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 13:47:16.670828 15872 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 13:47:16.671399 15872 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 13:47:16.671442 15872 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 13:47:16.671473 15872 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 13:47:16.671489 15872 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 24 13:47:16 dist-test-slave-sjl2 krb5kdc[15612](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257636, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.149.194@KRBTEST.COM for kudu/127.13.149.254@KRBTEST.COM
I20260924 13:47:16.678280 15872 rpc_server.cc:307] RPC server started. Bound to: 127.13.149.194:43723
I20260924 13:47:16.678406 16005 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.149.194:43723 every 8 connection(s)
I20260924 13:47:16.678673 15872 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790257629041645-13911-0/minicluster-data/ts-1/data/info.pb
I20260924 13:47:16.680860 15709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:16.671836 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.149.194:38899 (local address 127.13.149.254:40663)
0924 13:47:16.671899 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0924 13:47:16.671902 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 13:47:16.672160 (+ 258us) server_negotiation.cc:407] Connection header received
0924 13:47:16.672651 (+ 491us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:47:16.672653 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:47:16.672666 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:47:16.672742 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:47:16.673682 (+ 940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:16.674290 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:16.675340 (+ 1050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:16.675547 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:16.675602 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:47:16.678214 (+ 2612us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:47:16.678217 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:47:16.678217 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 13:47:16.678227 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:47:16.679671 (+ 1444us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:47:16.680064 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:47:16.680065 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:47:16.680065 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:47:16.680115 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:47:16.680303 (+ 188us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:47:16.680303 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:47:16.680303 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:47:16.680408 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:47:16.680454 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0924 13:47:16.680689 (+ 235us) server_negotiation.cc:299] Negotiation successful
0924 13:47:16.680760 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260924 13:47:16.681030 15896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:16.671865 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:40663 (local address 127.13.149.194:38899)
0924 13:47:16.672028 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0924 13:47:16.672069 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0924 13:47:16.672549 (+ 480us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:47:16.672846 (+ 297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:47:16.672851 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:47:16.673259 (+ 408us) client_negotiation.cc:189] Negotiated authn=SASL
0924 13:47:16.673574 (+ 315us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:47:16.673581 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:47:16.674389 (+ 808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:47:16.674390 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:47:16.675251 (+ 861us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:47:16.675254 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:47:16.675632 (+ 378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:47:16.675633 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:47:16.675791 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:47:16.676382 (+ 591us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 13:47:16.676392 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0924 13:47:16.678070 (+ 1678us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 13:47:16.679764 (+ 1694us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 13:47:16.679771 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 13:47:16.679772 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 13:47:16.679997 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 13:47:16.680154 (+ 157us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 13:47:16.680155 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 13:47:16.680155 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 13:47:16.680261 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 13:47:16.680454 (+ 193us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 13:47:16.680458 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 13:47:16.680585 (+ 127us) client_negotiation.cc:769] Sending connection context
0924 13:47:16.680668 (+ 83us) client_negotiation.cc:240] Negotiation successful
0924 13:47:16.680802 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":87,"threads_started":1}
I20260924 13:47:16.681516 16006 heartbeater.cc:347] Connected to a master server at 127.13.149.254:40663
I20260924 13:47:16.681604 16006 heartbeater.cc:464] Registering TS with master...
I20260924 13:47:16.681833 16006 heartbeater.cc:511] Master 127.13.149.254:40663 requested a full tablet report, sending...
I20260924 13:47:16.682188 15658 ts_manager.cc:194] Registered new tserver with Master: 19e79fbaeb2f4d8f9e4cb95bc73df894 (127.13.149.194:43723)
I20260924 13:47:16.682693 15658 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.149.194@KRBTEST.COM'} at 127.13.149.194:38899
I20260924 13:47:16.683064 13911 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJN9WRF/build/release/bin/kudu as pid 15872
I20260924 13:47:16.683154 13911 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790257629041645-13911-0/minicluster-data/ts-1/wal/instance
I20260924 13:47:16.683332 13911 external_mini_cluster.cc:1428] Setting key 077c788d5434c3833eb09a0a4c6cc85c
I20260924 13:47:16.688961 16012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:16.682505 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:40663 (local address 127.13.149.194:33035)
0924 13:47:16.682619 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0924 13:47:16.682630 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0924 13:47:16.682693 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:47:16.683508 (+ 815us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:47:16.683510 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:47:16.683692 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0924 13:47:16.683755 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:47:16.683757 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:47:16.684574 (+ 817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:47:16.684576 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:47:16.685152 (+ 576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:47:16.685155 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:47:16.685498 (+ 343us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:47:16.685499 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:47:16.685576 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:47:16.685996 (+ 420us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 13:47:16.686001 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 13:47:16.686323 (+ 322us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 13:47:16.687778 (+ 1455us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 13:47:16.687780 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 13:47:16.687780 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 13:47:16.687951 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 13:47:16.688124 (+ 173us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 13:47:16.688125 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 13:47:16.688126 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 13:47:16.688190 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 13:47:16.688475 (+ 285us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 13:47:16.688476 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 13:47:16.688531 (+ 55us) client_negotiation.cc:769] Sending connection context
0924 13:47:16.688592 (+ 61us) client_negotiation.cc:240] Negotiation successful
0924 13:47:16.688795 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":86,"thread_start_us":59,"threads_started":1}
I20260924 13:47:16.688910 15709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:16.682580 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.149.194:33035 (local address 127.13.149.254:40663)
0924 13:47:16.682621 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0924 13:47:16.682623 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:47:16.682677 (+ 54us) server_negotiation.cc:407] Connection header received
0924 13:47:16.682790 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:47:16.682791 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:47:16.682817 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:47:16.682869 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:47:16.683829 (+ 960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:16.684473 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:16.685238 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:16.685440 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:16.685494 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:47:16.686413 (+ 919us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:47:16.686414 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:47:16.686415 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 13:47:16.686422 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:47:16.687686 (+ 1264us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:47:16.688015 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:47:16.688017 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:47:16.688017 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:47:16.688070 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:47:16.688248 (+ 178us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:47:16.688249 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:47:16.688249 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:47:16.688411 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:47:16.688472 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0924 13:47:16.688623 (+ 151us) server_negotiation.cc:299] Negotiation successful
0924 13:47:16.688809 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
WARNING: no policy specified for kudu/127.13.149.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.149.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.149.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.195.keytab.
Entry for principal kudu/127.13.149.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.195.keytab.
I20260924 13:47:16.709980 13911 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJN9WRF/build/release/bin/kudu
/tmp/dist-test-taskJN9WRF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790257629041645-13911-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790257629041645-13911-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790257629041645-13911-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790257629041645-13911-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.149.195:0
--local_ip_for_outbound_sockets=127.13.149.195
--webserver_interface=127.13.149.195
--webserver_port=0
--tserver_master_addrs=127.13.149.254:40663
--keytab_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.195.keytab
--principal=kudu/127.13.149.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.149.212:42859
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790257629041645-13911-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790257629041645-13911-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790257629041645-13911-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 13:47:16.796312 16015 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 13:47:16.796552 16015 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 13:47:16.796581 16015 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 13:47:16.797814 16015 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 13:47:16.797863 16015 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 13:47:16.797884 16015 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260924 13:47:16.797895 16015 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260924 13:47:16.797911 16015 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.149.195
I20260924 13:47:16.799489 16015 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.149.212:42859
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790257629041645-13911-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790257629041645-13911-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.195.keytab
--principal=kudu/127.13.149.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.149.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790257629041645-13911-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.149.195
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.13.149.254:40663
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16015
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.149.195
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790257629041645-13911-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 12:29:37 UTC on ccycloud.quasar-iqzbxn.root.comops.site
build id 4670106
I20260924 13:47:16.799731 16015 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 13:47:16.799928 16015 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 13:47:16.805477 16020 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 13:47:16.805521 16015 server_base.cc:1034] running on GCE node
W20260924 13:47:16.805487 16023 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 13:47:16.805486 16021 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 13:47:16.805934 16015 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 13:47:16.806203 16015 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 13:47:16.807356 16015 hybrid_clock.cc:648] HybridClock initialized: now 1790257636807336 us; error 34 us; skew 500 ppm
Sep 24 13:47:16 dist-test-slave-sjl2 krb5kdc[15612](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257636, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.149.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 13:47:16.809876 16015 init.cc:377] Logged in from keytab as kudu/127.13.149.195@KRBTEST.COM (short username kudu)
I20260924 13:47:16.810365 16015 webserver.cc:460] Webserver started at http://127.13.149.195:42825/ using document root <none> and password file <none>
I20260924 13:47:16.810575 16015 fs_manager.cc:359] Metadata directory not provided
I20260924 13:47:16.810618 16015 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 13:47:16.810751 16015 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 13:47:16.811852 16015 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790257629041645-13911-0/minicluster-data/ts-2/data/instance:
uuid: "b0580d290b47438a9c7cace92e848865"
format_stamp: "Formatted at 2026-09-24 13:47:16 on dist-test-slave-sjl2"
server_key: "71c68ad779198cd603718a5023384455"
server_key_iv: "3c1c26d84590d32d7f71f3f8128d10c5"
server_key_version: "encryptionkey@0"
I20260924 13:47:16.812186 16015 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790257629041645-13911-0/minicluster-data/ts-2/wal/instance:
uuid: "b0580d290b47438a9c7cace92e848865"
format_stamp: "Formatted at 2026-09-24 13:47:16 on dist-test-slave-sjl2"
server_key: "71c68ad779198cd603718a5023384455"
server_key_iv: "3c1c26d84590d32d7f71f3f8128d10c5"
server_key_version: "encryptionkey@0"
I20260924 13:47:16.813975 16015 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.003s
I20260924 13:47:16.814776 16030 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 13:47:16.814963 16015 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260924 13:47:16.815043 16015 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790257629041645-13911-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790257629041645-13911-0/minicluster-data/ts-2/wal
uuid: "b0580d290b47438a9c7cace92e848865"
format_stamp: "Formatted at 2026-09-24 13:47:16 on dist-test-slave-sjl2"
server_key: "71c68ad779198cd603718a5023384455"
server_key_iv: "3c1c26d84590d32d7f71f3f8128d10c5"
server_key_version: "encryptionkey@0"
I20260924 13:47:16.815107 16015 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790257629041645-13911-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790257629041645-13911-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790257629041645-13911-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 13:47:16.848603 16015 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 13:47:16.850807 16015 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 13:47:16.850926 16015 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 13:47:16.851389 16015 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 13:47:16.851432 16015 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 13:47:16.851455 16015 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 13:47:16.851467 16015 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 24 13:47:16 dist-test-slave-sjl2 krb5kdc[15612](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257636, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.149.195@KRBTEST.COM for kudu/127.13.149.254@KRBTEST.COM
I20260924 13:47:16.858361 16015 rpc_server.cc:307] RPC server started. Bound to: 127.13.149.195:35443
I20260924 13:47:16.858421 16146 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.149.195:35443 every 8 connection(s)
I20260924 13:47:16.858778 16015 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790257629041645-13911-0/minicluster-data/ts-2/data/info.pb
I20260924 13:47:16.860962 15709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:16.851893 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.149.195:60945 (local address 127.13.149.254:40663)
0924 13:47:16.851934 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0924 13:47:16.851936 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:47:16.852184 (+ 248us) server_negotiation.cc:407] Connection header received
0924 13:47:16.852704 (+ 520us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:47:16.852705 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:47:16.852718 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:47:16.852777 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:47:16.853531 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:16.854028 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:16.855058 (+ 1030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:16.855249 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:16.855328 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:47:16.858023 (+ 2695us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:47:16.858025 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:47:16.858025 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 13:47:16.858037 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:47:16.859557 (+ 1520us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:47:16.859940 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:47:16.859940 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:47:16.859941 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:47:16.859989 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:47:16.860237 (+ 248us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:47:16.860238 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:47:16.860238 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:47:16.860372 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:47:16.860482 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0924 13:47:16.860791 (+ 309us) server_negotiation.cc:299] Negotiation successful
0924 13:47:16.860868 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260924 13:47:16.861194 16039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:16.851898 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:40663 (local address 127.13.149.195:60945)
0924 13:47:16.852069 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0924 13:47:16.852106 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0924 13:47:16.852602 (+ 496us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:47:16.852898 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:47:16.852903 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:47:16.853216 (+ 313us) client_negotiation.cc:189] Negotiated authn=SASL
0924 13:47:16.853432 (+ 216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:47:16.853437 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:47:16.854113 (+ 676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:47:16.854114 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:47:16.854970 (+ 856us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:47:16.854972 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:47:16.855346 (+ 374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:47:16.855347 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:47:16.855500 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:47:16.856069 (+ 569us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 13:47:16.856082 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0924 13:47:16.857908 (+ 1826us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 13:47:16.859649 (+ 1741us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 13:47:16.859654 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 13:47:16.859656 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 13:47:16.859880 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 13:47:16.860055 (+ 175us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 13:47:16.860055 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 13:47:16.860056 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 13:47:16.860155 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 13:47:16.860490 (+ 335us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 13:47:16.860497 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 13:47:16.860668 (+ 171us) client_negotiation.cc:769] Sending connection context
0924 13:47:16.860758 (+ 90us) client_negotiation.cc:240] Negotiation successful
0924 13:47:16.860963 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":95,"threads_started":1}
I20260924 13:47:16.861694 16147 heartbeater.cc:347] Connected to a master server at 127.13.149.254:40663
I20260924 13:47:16.861795 16147 heartbeater.cc:464] Registering TS with master...
I20260924 13:47:16.861979 16147 heartbeater.cc:511] Master 127.13.149.254:40663 requested a full tablet report, sending...
I20260924 13:47:16.862344 15658 ts_manager.cc:194] Registered new tserver with Master: b0580d290b47438a9c7cace92e848865 (127.13.149.195:35443)
I20260924 13:47:16.862807 15658 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.149.195@KRBTEST.COM'} at 127.13.149.195:60945
I20260924 13:47:16.865657 13911 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJN9WRF/build/release/bin/kudu as pid 16015
I20260924 13:47:16.865758 13911 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790257629041645-13911-0/minicluster-data/ts-2/wal/instance
I20260924 13:47:16.865945 13911 external_mini_cluster.cc:1428] Setting key 5beca0fd5333a6fc295ba07a09126e7f
I20260924 13:47:16.867296 13911 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260924 13:47:16.868736 16153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:16.862690 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:40663 (local address 127.13.149.195:52317)
0924 13:47:16.862823 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0924 13:47:16.862833 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 13:47:16.862890 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:47:16.863092 (+ 202us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:47:16.863093 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:47:16.863281 (+ 188us) client_negotiation.cc:189] Negotiated authn=SASL
0924 13:47:16.863347 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:47:16.863348 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:47:16.864009 (+ 661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:47:16.864010 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:47:16.864527 (+ 517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:47:16.864528 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:47:16.864870 (+ 342us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:47:16.864871 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:47:16.864980 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:47:16.865382 (+ 402us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 13:47:16.865386 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0924 13:47:16.865860 (+ 474us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 13:47:16.867655 (+ 1795us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 13:47:16.867656 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 13:47:16.867657 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 13:47:16.867842 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 13:47:16.868033 (+ 191us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 13:47:16.868033 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 13:47:16.868034 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 13:47:16.868100 (+ 66us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 13:47:16.868369 (+ 269us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 13:47:16.868370 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 13:47:16.868425 (+ 55us) client_negotiation.cc:769] Sending connection context
0924 13:47:16.868492 (+ 67us) client_negotiation.cc:240] Negotiation successful
0924 13:47:16.868612 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":84,"threads_started":1}
I20260924 13:47:16.868709 15709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:16.862765 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.149.195:52317 (local address 127.13.149.254:40663)
0924 13:47:16.862809 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0924 13:47:16.862811 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:47:16.862891 (+ 80us) server_negotiation.cc:407] Connection header received
0924 13:47:16.863021 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:47:16.863022 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:47:16.863032 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:47:16.863085 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:47:16.863422 (+ 337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:16.863919 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:16.864604 (+ 685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:16.864809 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:16.864864 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:47:16.865974 (+ 1110us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:47:16.865976 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:47:16.865976 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 13:47:16.865984 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:47:16.867569 (+ 1585us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:47:16.867911 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:47:16.867912 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:47:16.867912 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:47:16.867976 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:47:16.868174 (+ 198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:47:16.868175 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:47:16.868175 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:47:16.868306 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:47:16.868371 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0924 13:47:16.868523 (+ 152us) server_negotiation.cc:299] Negotiation successful
0924 13:47:16.868609 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
Sep 24 13:47:16 dist-test-slave-sjl2 krb5kdc[15612](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257636, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 24 13:47:16 dist-test-slave-sjl2 krb5kdc[15612](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257636, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.149.254@KRBTEST.COM
I20260924 13:47:16.891016 15709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:16.883528 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42852 (local address 127.13.149.254:40663)
0924 13:47:16.883598 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0924 13:47:16.883601 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 13:47:16.883610 (+ 9us) server_negotiation.cc:407] Connection header received
0924 13:47:16.883636 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:47:16.883637 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:47:16.883645 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:47:16.883699 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:47:16.884290 (+ 591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:16.884795 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:16.885620 (+ 825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:16.885819 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:16.885876 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:47:16.888166 (+ 2290us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:47:16.888167 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:47:16.888168 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 13:47:16.888176 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:47:16.889670 (+ 1494us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:47:16.890079 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:47:16.890080 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:47:16.890080 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:47:16.890141 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:47:16.890423 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:47:16.890424 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:47:16.890424 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:47:16.890565 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:47:16.890654 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0924 13:47:16.890859 (+ 205us) server_negotiation.cc:299] Negotiation successful
0924 13:47:16.890939 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260924 13:47:16.892900 15658 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42852:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 13:47:16.894618 15658 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 13:47:16.901654 16167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:16.897283 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53148 (local address 127.13.149.194:43723)
0924 13:47:16.898100 (+ 817us) server_negotiation.cc:206] Beginning negotiation
0924 13:47:16.898105 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 13:47:16.898115 (+ 10us) server_negotiation.cc:407] Connection header received
0924 13:47:16.898150 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:47:16.898152 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:47:16.898176 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:47:16.898270 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 13:47:16.898376 (+ 106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:16.899071 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:16.900531 (+ 1460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:16.901269 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:16.901362 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:47:16.901414 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0924 13:47:16.901520 (+ 106us) server_negotiation.cc:299] Negotiation successful
0924 13:47:16.901558 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":770,"thread_start_us":80,"threads_started":1}
I20260924 13:47:16.901683 16164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:16.897130 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.194:43723 (local address 127.0.0.1:53148)
0924 13:47:16.897410 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0924 13:47:16.897420 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 13:47:16.897466 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:47:16.898274 (+ 808us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:47:16.898275 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:47:16.898277 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 13:47:16.898324 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:47:16.898326 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:47:16.899162 (+ 836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:47:16.899163 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:47:16.900444 (+ 1281us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:47:16.900447 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:47:16.901367 (+ 920us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:47:16.901368 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:47:16.901441 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:47:16.901451 (+ 10us) client_negotiation.cc:769] Sending connection context
0924 13:47:16.901503 (+ 52us) client_negotiation.cc:240] Negotiation successful
0924 13:47:16.901548 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":266,"mutex_wait_us":77,"thread_start_us":95,"threads_started":1}
I20260924 13:47:16.901573 16166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:16.897303 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.195:35443 (local address 127.0.0.1:46412)
0924 13:47:16.897898 (+ 595us) negotiation.cc:107] Waiting for socket to connect
0924 13:47:16.897909 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0924 13:47:16.897972 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:47:16.898220 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:47:16.898221 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:47:16.898223 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 13:47:16.898282 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:47:16.898284 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:47:16.899396 (+ 1112us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:47:16.899397 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:47:16.900310 (+ 913us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:47:16.900312 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:47:16.901248 (+ 936us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:47:16.901249 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:47:16.901335 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:47:16.901351 (+ 16us) client_negotiation.cc:769] Sending connection context
0924 13:47:16.901416 (+ 65us) client_negotiation.cc:240] Negotiation successful
0924 13:47:16.901477 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":566,"thread_start_us":60,"threads_started":1}
I20260924 13:47:16.901765 16168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:16.897298 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46830 (local address 127.13.149.193:40235)
0924 13:47:16.897598 (+ 300us) server_negotiation.cc:206] Beginning negotiation
0924 13:47:16.897603 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 13:47:16.897612 (+ 9us) server_negotiation.cc:407] Connection header received
0924 13:47:16.897663 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:47:16.897665 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:47:16.897678 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:47:16.897744 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 13:47:16.897914 (+ 170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:16.899058 (+ 1144us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:16.900656 (+ 1598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:16.901350 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:16.901419 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:47:16.901471 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0924 13:47:16.901636 (+ 165us) server_negotiation.cc:299] Negotiation successful
0924 13:47:16.901677 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":263,"thread_start_us":92,"threads_started":1}
I20260924 13:47:16.901983 16169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:16.897562 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46412 (local address 127.13.149.195:35443)
0924 13:47:16.897710 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0924 13:47:16.897715 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 13:47:16.898026 (+ 311us) server_negotiation.cc:407] Connection header received
0924 13:47:16.898067 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:47:16.898069 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:47:16.898083 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:47:16.898154 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 13:47:16.898354 (+ 200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:16.899324 (+ 970us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:16.900384 (+ 1060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:16.901170 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:16.901258 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:47:16.901321 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0924 13:47:16.901837 (+ 516us) server_negotiation.cc:299] Negotiation successful
0924 13:47:16.901887 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":118,"thread_start_us":76,"threads_started":1}
I20260924 13:47:16.902387 16165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:16.897115 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.193:40235 (local address 127.0.0.1:46830)
0924 13:47:16.897315 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0924 13:47:16.897365 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0924 13:47:16.897438 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:47:16.897760 (+ 322us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:47:16.897763 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:47:16.897765 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 13:47:16.897838 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:47:16.897840 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:47:16.899507 (+ 1667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:47:16.899508 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:47:16.900584 (+ 1076us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:47:16.900585 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:47:16.901502 (+ 917us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:47:16.901503 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:47:16.901562 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:47:16.901565 (+ 3us) client_negotiation.cc:769] Sending connection context
0924 13:47:16.901614 (+ 49us) client_negotiation.cc:240] Negotiation successful
0924 13:47:16.901653 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":47,"threads_started":1}
I20260924 13:47:16.902617 15938 tablet_service.cc:1467] Processing CreateTablet for tablet 31b03e63e4b440b5b6836b0cca1a7f1a (DEFAULT_TABLE table=test-table [id=28d8485f7a33466db4d76b05cfe8ee5f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 13:47:16.902621 16081 tablet_service.cc:1467] Processing CreateTablet for tablet 31b03e63e4b440b5b6836b0cca1a7f1a (DEFAULT_TABLE table=test-table [id=28d8485f7a33466db4d76b05cfe8ee5f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 13:47:16.902953 16081 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 31b03e63e4b440b5b6836b0cca1a7f1a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 13:47:16.902967 15938 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 31b03e63e4b440b5b6836b0cca1a7f1a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 13:47:16.903072 15797 tablet_service.cc:1467] Processing CreateTablet for tablet 31b03e63e4b440b5b6836b0cca1a7f1a (DEFAULT_TABLE table=test-table [id=28d8485f7a33466db4d76b05cfe8ee5f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 13:47:16.903390 15797 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 31b03e63e4b440b5b6836b0cca1a7f1a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 13:47:16.905668 16170 tablet_bootstrap.cc:492] T 31b03e63e4b440b5b6836b0cca1a7f1a P b0580d290b47438a9c7cace92e848865: Bootstrap starting.
I20260924 13:47:16.906517 16170 tablet_bootstrap.cc:654] T 31b03e63e4b440b5b6836b0cca1a7f1a P b0580d290b47438a9c7cace92e848865: Neither blocks nor log segments found. Creating new log.
I20260924 13:47:16.906598 16171 tablet_bootstrap.cc:492] T 31b03e63e4b440b5b6836b0cca1a7f1a P 19e79fbaeb2f4d8f9e4cb95bc73df894: Bootstrap starting.
I20260924 13:47:16.906824 16170 log.cc:824] T 31b03e63e4b440b5b6836b0cca1a7f1a P b0580d290b47438a9c7cace92e848865: Log is configured to *not* fsync() on all Append() calls
I20260924 13:47:16.907153 16172 tablet_bootstrap.cc:492] T 31b03e63e4b440b5b6836b0cca1a7f1a P 58c281726a7d46c2b6c754c8a4fe5bf4: Bootstrap starting.
I20260924 13:47:16.907271 16171 tablet_bootstrap.cc:654] T 31b03e63e4b440b5b6836b0cca1a7f1a P 19e79fbaeb2f4d8f9e4cb95bc73df894: Neither blocks nor log segments found. Creating new log.
I20260924 13:47:16.907586 16171 log.cc:824] T 31b03e63e4b440b5b6836b0cca1a7f1a P 19e79fbaeb2f4d8f9e4cb95bc73df894: Log is configured to *not* fsync() on all Append() calls
I20260924 13:47:16.907838 16170 tablet_bootstrap.cc:492] T 31b03e63e4b440b5b6836b0cca1a7f1a P b0580d290b47438a9c7cace92e848865: No bootstrap required, opened a new log
I20260924 13:47:16.907907 16170 ts_tablet_manager.cc:1397] T 31b03e63e4b440b5b6836b0cca1a7f1a P b0580d290b47438a9c7cace92e848865: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260924 13:47:16.908018 16172 tablet_bootstrap.cc:654] T 31b03e63e4b440b5b6836b0cca1a7f1a P 58c281726a7d46c2b6c754c8a4fe5bf4: Neither blocks nor log segments found. Creating new log.
I20260924 13:47:16.908309 16172 log.cc:824] T 31b03e63e4b440b5b6836b0cca1a7f1a P 58c281726a7d46c2b6c754c8a4fe5bf4: Log is configured to *not* fsync() on all Append() calls
I20260924 13:47:16.908331 16171 tablet_bootstrap.cc:492] T 31b03e63e4b440b5b6836b0cca1a7f1a P 19e79fbaeb2f4d8f9e4cb95bc73df894: No bootstrap required, opened a new log
I20260924 13:47:16.908388 16171 ts_tablet_manager.cc:1397] T 31b03e63e4b440b5b6836b0cca1a7f1a P 19e79fbaeb2f4d8f9e4cb95bc73df894: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260924 13:47:16.909096 16172 tablet_bootstrap.cc:492] T 31b03e63e4b440b5b6836b0cca1a7f1a P 58c281726a7d46c2b6c754c8a4fe5bf4: No bootstrap required, opened a new log
I20260924 13:47:16.909155 16172 ts_tablet_manager.cc:1397] T 31b03e63e4b440b5b6836b0cca1a7f1a P 58c281726a7d46c2b6c754c8a4fe5bf4: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260924 13:47:16.909940 16170 raft_consensus.cc:348] T 31b03e63e4b440b5b6836b0cca1a7f1a P b0580d290b47438a9c7cace92e848865 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58c281726a7d46c2b6c754c8a4fe5bf4" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 40235 } } peers { permanent_uuid: "19e79fbaeb2f4d8f9e4cb95bc73df894" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 43723 } } peers { permanent_uuid: "b0580d290b47438a9c7cace92e848865" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 35443 } }
I20260924 13:47:16.910122 16170 raft_consensus.cc:374] T 31b03e63e4b440b5b6836b0cca1a7f1a P b0580d290b47438a9c7cace92e848865 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 13:47:16.910154 16170 raft_consensus.cc:729] T 31b03e63e4b440b5b6836b0cca1a7f1a P b0580d290b47438a9c7cace92e848865 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b0580d290b47438a9c7cace92e848865, State: Initialized, Role: FOLLOWER
I20260924 13:47:16.910286 16170 consensus_queue.cc:260] T 31b03e63e4b440b5b6836b0cca1a7f1a P b0580d290b47438a9c7cace92e848865 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58c281726a7d46c2b6c754c8a4fe5bf4" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 40235 } } peers { permanent_uuid: "19e79fbaeb2f4d8f9e4cb95bc73df894" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 43723 } } peers { permanent_uuid: "b0580d290b47438a9c7cace92e848865" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 35443 } }
I20260924 13:47:16.910352 16171 raft_consensus.cc:348] T 31b03e63e4b440b5b6836b0cca1a7f1a P 19e79fbaeb2f4d8f9e4cb95bc73df894 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58c281726a7d46c2b6c754c8a4fe5bf4" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 40235 } } peers { permanent_uuid: "19e79fbaeb2f4d8f9e4cb95bc73df894" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 43723 } } peers { permanent_uuid: "b0580d290b47438a9c7cace92e848865" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 35443 } }
I20260924 13:47:16.910496 16171 raft_consensus.cc:374] T 31b03e63e4b440b5b6836b0cca1a7f1a P 19e79fbaeb2f4d8f9e4cb95bc73df894 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 13:47:16.910533 16171 raft_consensus.cc:729] T 31b03e63e4b440b5b6836b0cca1a7f1a P 19e79fbaeb2f4d8f9e4cb95bc73df894 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 19e79fbaeb2f4d8f9e4cb95bc73df894, State: Initialized, Role: FOLLOWER
I20260924 13:47:16.910548 16170 ts_tablet_manager.cc:1428] T 31b03e63e4b440b5b6836b0cca1a7f1a P b0580d290b47438a9c7cace92e848865: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260924 13:47:16.910658 16147 heartbeater.cc:503] Master 127.13.149.254:40663 was elected leader, sending a full tablet report...
I20260924 13:47:16.910640 16171 consensus_queue.cc:260] T 31b03e63e4b440b5b6836b0cca1a7f1a P 19e79fbaeb2f4d8f9e4cb95bc73df894 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58c281726a7d46c2b6c754c8a4fe5bf4" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 40235 } } peers { permanent_uuid: "19e79fbaeb2f4d8f9e4cb95bc73df894" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 43723 } } peers { permanent_uuid: "b0580d290b47438a9c7cace92e848865" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 35443 } }
I20260924 13:47:16.910701 16172 raft_consensus.cc:348] T 31b03e63e4b440b5b6836b0cca1a7f1a P 58c281726a7d46c2b6c754c8a4fe5bf4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58c281726a7d46c2b6c754c8a4fe5bf4" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 40235 } } peers { permanent_uuid: "19e79fbaeb2f4d8f9e4cb95bc73df894" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 43723 } } peers { permanent_uuid: "b0580d290b47438a9c7cace92e848865" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 35443 } }
I20260924 13:47:16.910856 16172 raft_consensus.cc:374] T 31b03e63e4b440b5b6836b0cca1a7f1a P 58c281726a7d46c2b6c754c8a4fe5bf4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 13:47:16.910887 16171 ts_tablet_manager.cc:1428] T 31b03e63e4b440b5b6836b0cca1a7f1a P 19e79fbaeb2f4d8f9e4cb95bc73df894: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260924 13:47:16.910898 16172 raft_consensus.cc:729] T 31b03e63e4b440b5b6836b0cca1a7f1a P 58c281726a7d46c2b6c754c8a4fe5bf4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 58c281726a7d46c2b6c754c8a4fe5bf4, State: Initialized, Role: FOLLOWER
I20260924 13:47:16.910998 16006 heartbeater.cc:503] Master 127.13.149.254:40663 was elected leader, sending a full tablet report...
I20260924 13:47:16.911001 16172 consensus_queue.cc:260] T 31b03e63e4b440b5b6836b0cca1a7f1a P 58c281726a7d46c2b6c754c8a4fe5bf4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58c281726a7d46c2b6c754c8a4fe5bf4" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 40235 } } peers { permanent_uuid: "19e79fbaeb2f4d8f9e4cb95bc73df894" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 43723 } } peers { permanent_uuid: "b0580d290b47438a9c7cace92e848865" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 35443 } }
I20260924 13:47:16.911248 16172 ts_tablet_manager.cc:1428] T 31b03e63e4b440b5b6836b0cca1a7f1a P 58c281726a7d46c2b6c754c8a4fe5bf4: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260924 13:47:16.911288 15863 heartbeater.cc:503] Master 127.13.149.254:40663 was elected leader, sending a full tablet report...
W20260924 13:47:16.929291 16007 tablet.cc:2369] T 31b03e63e4b440b5b6836b0cca1a7f1a P 19e79fbaeb2f4d8f9e4cb95bc73df894: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260924 13:47:16.975108 16177 raft_consensus.cc:482] T 31b03e63e4b440b5b6836b0cca1a7f1a P 19e79fbaeb2f4d8f9e4cb95bc73df894 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 13:47:16.975252 16177 raft_consensus.cc:504] T 31b03e63e4b440b5b6836b0cca1a7f1a P 19e79fbaeb2f4d8f9e4cb95bc73df894 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58c281726a7d46c2b6c754c8a4fe5bf4" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 40235 } } peers { permanent_uuid: "19e79fbaeb2f4d8f9e4cb95bc73df894" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 43723 } } peers { permanent_uuid: "b0580d290b47438a9c7cace92e848865" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 35443 } }
I20260924 13:47:16.975575 16177 leader_election.cc:288] T 31b03e63e4b440b5b6836b0cca1a7f1a P 19e79fbaeb2f4d8f9e4cb95bc73df894 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 58c281726a7d46c2b6c754c8a4fe5bf4 (127.13.149.193:40235), b0580d290b47438a9c7cace92e848865 (127.13.149.195:35443)
I20260924 13:47:16.978615 16169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:16.976003 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.149.194:50253 (local address 127.13.149.195:35443)
0924 13:47:16.976051 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0924 13:47:16.976053 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:47:16.976095 (+ 42us) server_negotiation.cc:407] Connection header received
0924 13:47:16.976185 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:47:16.976186 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:47:16.976195 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:47:16.976246 (+ 51us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 13:47:16.976435 (+ 189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:16.976981 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:16.977907 (+ 926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:16.978281 (+ 374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:16.978350 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:47:16.978358 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0924 13:47:16.978497 (+ 139us) server_negotiation.cc:299] Negotiation successful
0924 13:47:16.978538 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260924 13:47:16.978621 16168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:16.975904 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.149.194:38185 (local address 127.13.149.193:40235)
0924 13:47:16.975979 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0924 13:47:16.975981 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:47:16.975988 (+ 7us) server_negotiation.cc:407] Connection header received
0924 13:47:16.976016 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:47:16.976017 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:47:16.976026 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:47:16.976087 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 13:47:16.976258 (+ 171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:16.976790 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:16.977883 (+ 1093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:16.978261 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:16.978321 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:47:16.978329 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0924 13:47:16.978500 (+ 171us) server_negotiation.cc:299] Negotiation successful
0924 13:47:16.978539 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260924 13:47:16.978645 15896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:16.975792 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.193:40235 (local address 127.13.149.194:38185)
0924 13:47:16.975855 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0924 13:47:16.975870 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0924 13:47:16.975938 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:47:16.976109 (+ 171us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:47:16.976110 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:47:16.976113 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 13:47:16.976188 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:47:16.976190 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:47:16.976881 (+ 691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:47:16.976883 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:47:16.977774 (+ 891us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:47:16.977778 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:47:16.978347 (+ 569us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:47:16.978348 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:47:16.978431 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:47:16.978434 (+ 3us) client_negotiation.cc:769] Sending connection context
0924 13:47:16.978480 (+ 46us) client_negotiation.cc:240] Negotiation successful
0924 13:47:16.978547 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36,"mutex_wait_us":39}
I20260924 13:47:16.978657 16179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:16.975889 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.195:35443 (local address 127.13.149.194:50253)
0924 13:47:16.976046 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0924 13:47:16.976054 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0924 13:47:16.976100 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:47:16.976269 (+ 169us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:47:16.976270 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:47:16.976271 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 13:47:16.976343 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:47:16.976345 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:47:16.977075 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:47:16.977076 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:47:16.977816 (+ 740us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:47:16.977818 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:47:16.978362 (+ 544us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:47:16.978363 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:47:16.978429 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:47:16.978433 (+ 4us) client_negotiation.cc:769] Sending connection context
0924 13:47:16.978482 (+ 49us) client_negotiation.cc:240] Negotiation successful
0924 13:47:16.978539 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"mutex_wait_us":5,"thread_start_us":115,"threads_started":1}
I20260924 13:47:16.979063 15817 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "31b03e63e4b440b5b6836b0cca1a7f1a" candidate_uuid: "19e79fbaeb2f4d8f9e4cb95bc73df894" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58c281726a7d46c2b6c754c8a4fe5bf4" is_pre_election: true
I20260924 13:47:16.979200 16101 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "31b03e63e4b440b5b6836b0cca1a7f1a" candidate_uuid: "19e79fbaeb2f4d8f9e4cb95bc73df894" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b0580d290b47438a9c7cace92e848865" is_pre_election: true
I20260924 13:47:16.979249 15817 raft_consensus.cc:2454] T 31b03e63e4b440b5b6836b0cca1a7f1a P 58c281726a7d46c2b6c754c8a4fe5bf4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19e79fbaeb2f4d8f9e4cb95bc73df894 in term 0.
I20260924 13:47:16.979334 16101 raft_consensus.cc:2454] T 31b03e63e4b440b5b6836b0cca1a7f1a P b0580d290b47438a9c7cace92e848865 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19e79fbaeb2f4d8f9e4cb95bc73df894 in term 0.
I20260924 13:47:16.979506 15890 leader_election.cc:302] T 31b03e63e4b440b5b6836b0cca1a7f1a P 19e79fbaeb2f4d8f9e4cb95bc73df894 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19e79fbaeb2f4d8f9e4cb95bc73df894, 58c281726a7d46c2b6c754c8a4fe5bf4; no voters:
I20260924 13:47:16.979640 16177 raft_consensus.cc:2781] T 31b03e63e4b440b5b6836b0cca1a7f1a P 19e79fbaeb2f4d8f9e4cb95bc73df894 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 13:47:16.979696 16177 raft_consensus.cc:482] T 31b03e63e4b440b5b6836b0cca1a7f1a P 19e79fbaeb2f4d8f9e4cb95bc73df894 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 13:47:16.979725 16177 raft_consensus.cc:3037] T 31b03e63e4b440b5b6836b0cca1a7f1a P 19e79fbaeb2f4d8f9e4cb95bc73df894 [term 0 FOLLOWER]: Advancing to term 1
I20260924 13:47:16.980609 16177 raft_consensus.cc:504] T 31b03e63e4b440b5b6836b0cca1a7f1a P 19e79fbaeb2f4d8f9e4cb95bc73df894 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58c281726a7d46c2b6c754c8a4fe5bf4" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 40235 } } peers { permanent_uuid: "19e79fbaeb2f4d8f9e4cb95bc73df894" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 43723 } } peers { permanent_uuid: "b0580d290b47438a9c7cace92e848865" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 35443 } }
I20260924 13:47:16.980731 16177 leader_election.cc:288] T 31b03e63e4b440b5b6836b0cca1a7f1a P 19e79fbaeb2f4d8f9e4cb95bc73df894 [CANDIDATE]: Term 1 election: Requested vote from peers 58c281726a7d46c2b6c754c8a4fe5bf4 (127.13.149.193:40235), b0580d290b47438a9c7cace92e848865 (127.13.149.195:35443)
I20260924 13:47:16.980942 15817 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "31b03e63e4b440b5b6836b0cca1a7f1a" candidate_uuid: "19e79fbaeb2f4d8f9e4cb95bc73df894" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58c281726a7d46c2b6c754c8a4fe5bf4"
I20260924 13:47:16.980966 16101 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "31b03e63e4b440b5b6836b0cca1a7f1a" candidate_uuid: "19e79fbaeb2f4d8f9e4cb95bc73df894" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b0580d290b47438a9c7cace92e848865"
I20260924 13:47:16.981029 15817 raft_consensus.cc:3037] T 31b03e63e4b440b5b6836b0cca1a7f1a P 58c281726a7d46c2b6c754c8a4fe5bf4 [term 0 FOLLOWER]: Advancing to term 1
I20260924 13:47:16.981056 16101 raft_consensus.cc:3037] T 31b03e63e4b440b5b6836b0cca1a7f1a P b0580d290b47438a9c7cace92e848865 [term 0 FOLLOWER]: Advancing to term 1
I20260924 13:47:16.981757 15817 raft_consensus.cc:2454] T 31b03e63e4b440b5b6836b0cca1a7f1a P 58c281726a7d46c2b6c754c8a4fe5bf4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19e79fbaeb2f4d8f9e4cb95bc73df894 in term 1.
I20260924 13:47:16.981775 16101 raft_consensus.cc:2454] T 31b03e63e4b440b5b6836b0cca1a7f1a P b0580d290b47438a9c7cace92e848865 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19e79fbaeb2f4d8f9e4cb95bc73df894 in term 1.
I20260924 13:47:16.981922 15890 leader_election.cc:302] T 31b03e63e4b440b5b6836b0cca1a7f1a P 19e79fbaeb2f4d8f9e4cb95bc73df894 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19e79fbaeb2f4d8f9e4cb95bc73df894, b0580d290b47438a9c7cace92e848865; no voters:
I20260924 13:47:16.982009 16177 raft_consensus.cc:2781] T 31b03e63e4b440b5b6836b0cca1a7f1a P 19e79fbaeb2f4d8f9e4cb95bc73df894 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 13:47:16.982182 16177 raft_consensus.cc:686] T 31b03e63e4b440b5b6836b0cca1a7f1a P 19e79fbaeb2f4d8f9e4cb95bc73df894 [term 1 LEADER]: Becoming Leader. State: Replica: 19e79fbaeb2f4d8f9e4cb95bc73df894, State: Running, Role: LEADER
I20260924 13:47:16.982271 16177 consensus_queue.cc:237] T 31b03e63e4b440b5b6836b0cca1a7f1a P 19e79fbaeb2f4d8f9e4cb95bc73df894 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58c281726a7d46c2b6c754c8a4fe5bf4" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 40235 } } peers { permanent_uuid: "19e79fbaeb2f4d8f9e4cb95bc73df894" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 43723 } } peers { permanent_uuid: "b0580d290b47438a9c7cace92e848865" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 35443 } }
I20260924 13:47:16.983067 15658 catalog_manager.cc:5495] T 31b03e63e4b440b5b6836b0cca1a7f1a P 19e79fbaeb2f4d8f9e4cb95bc73df894 reported cstate change: term changed from 0 to 1, leader changed from <none> to 19e79fbaeb2f4d8f9e4cb95bc73df894 (127.13.149.194). New cstate: current_term: 1 leader_uuid: "19e79fbaeb2f4d8f9e4cb95bc73df894" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58c281726a7d46c2b6c754c8a4fe5bf4" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 40235 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "19e79fbaeb2f4d8f9e4cb95bc73df894" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 43723 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b0580d290b47438a9c7cace92e848865" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 35443 } health_report { overall_health: UNKNOWN } } }
Sep 24 13:47:17 dist-test-slave-sjl2 krb5kdc[15612](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257636, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.149.254@KRBTEST.COM for kudu/127.13.149.254@KRBTEST.COM
I20260924 13:47:17.021073 15709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:17.013790 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42860 (local address 127.13.149.254:40663)
0924 13:47:17.013885 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0924 13:47:17.013888 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 13:47:17.013995 (+ 107us) server_negotiation.cc:407] Connection header received
0924 13:47:17.014068 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:47:17.014069 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:47:17.014077 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:47:17.014118 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:47:17.014635 (+ 517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:17.015366 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:17.016041 (+ 675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:17.016233 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:17.016290 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:47:17.018449 (+ 2159us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:47:17.018450 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:47:17.018451 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 13:47:17.018458 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:47:17.019951 (+ 1493us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:47:17.020356 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:47:17.020357 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:47:17.020358 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:47:17.020408 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:47:17.020557 (+ 149us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:47:17.020558 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:47:17.020559 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:47:17.020666 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:47:17.020728 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0924 13:47:17.020859 (+ 131us) server_negotiation.cc:299] Negotiation successful
0924 13:47:17.020964 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260924 13:47:17.021152 16187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:17.013722 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:40663 (local address 127.0.0.1:42860)
0924 13:47:17.013923 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0924 13:47:17.013935 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0924 13:47:17.014006 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:47:17.014128 (+ 122us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:47:17.014129 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:47:17.014456 (+ 327us) client_negotiation.cc:189] Negotiated authn=SASL
0924 13:47:17.014516 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:47:17.014520 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:47:17.015438 (+ 918us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:47:17.015439 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:47:17.015964 (+ 525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:47:17.015966 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:47:17.016302 (+ 336us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:47:17.016303 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:47:17.016351 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:47:17.016865 (+ 514us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 13:47:17.016870 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 13:47:17.018356 (+ 1486us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 13:47:17.020051 (+ 1695us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 13:47:17.020063 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 13:47:17.020066 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0924 13:47:17.020288 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 13:47:17.020460 (+ 172us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 13:47:17.020461 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 13:47:17.020461 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 13:47:17.020511 (+ 50us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 13:47:17.020719 (+ 208us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 13:47:17.020727 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 13:47:17.020775 (+ 48us) client_negotiation.cc:769] Sending connection context
0924 13:47:17.020834 (+ 59us) client_negotiation.cc:240] Negotiation successful
0924 13:47:17.021025 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":172,"mutex_wait_us":78,"thread_start_us":106,"threads_started":1}
I20260924 13:47:17.023260 15658 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.13.149.254@KRBTEST.COM'} at 127.0.0.1:42860:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260924 13:47:17.023480 15658 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 13:47:17.025106 15797 tablet_service.cc:1467] Processing CreateTablet for tablet b638038ba33c4697ba022eed3e8bbd64 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=b46a4b92d5eb45a8833b6402fbf7f712]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260924 13:47:17.025118 16081 tablet_service.cc:1467] Processing CreateTablet for tablet b638038ba33c4697ba022eed3e8bbd64 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=b46a4b92d5eb45a8833b6402fbf7f712]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260924 13:47:17.025298 15797 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b638038ba33c4697ba022eed3e8bbd64. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 13:47:17.025301 16081 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b638038ba33c4697ba022eed3e8bbd64. 1 dirs total, 0 dirs full, 0 dirs failed
W20260924 13:47:17.026245 15864 tablet.cc:2369] T 31b03e63e4b440b5b6836b0cca1a7f1a P 58c281726a7d46c2b6c754c8a4fe5bf4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260924 13:47:17.026502 16170 tablet_bootstrap.cc:492] T b638038ba33c4697ba022eed3e8bbd64 P b0580d290b47438a9c7cace92e848865: Bootstrap starting.
I20260924 13:47:17.026634 15938 tablet_service.cc:1467] Processing CreateTablet for tablet b638038ba33c4697ba022eed3e8bbd64 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=b46a4b92d5eb45a8833b6402fbf7f712]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260924 13:47:17.026880 15938 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b638038ba33c4697ba022eed3e8bbd64. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 13:47:17.027055 16170 tablet_bootstrap.cc:654] T b638038ba33c4697ba022eed3e8bbd64 P b0580d290b47438a9c7cace92e848865: Neither blocks nor log segments found. Creating new log.
I20260924 13:47:17.027419 16172 tablet_bootstrap.cc:492] T b638038ba33c4697ba022eed3e8bbd64 P 58c281726a7d46c2b6c754c8a4fe5bf4: Bootstrap starting.
I20260924 13:47:17.027742 16170 tablet_bootstrap.cc:492] T b638038ba33c4697ba022eed3e8bbd64 P b0580d290b47438a9c7cace92e848865: No bootstrap required, opened a new log
I20260924 13:47:17.027801 16170 ts_tablet_manager.cc:1397] T b638038ba33c4697ba022eed3e8bbd64 P b0580d290b47438a9c7cace92e848865: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260924 13:47:17.027833 16172 tablet_bootstrap.cc:654] T b638038ba33c4697ba022eed3e8bbd64 P 58c281726a7d46c2b6c754c8a4fe5bf4: Neither blocks nor log segments found. Creating new log.
I20260924 13:47:17.027961 16170 raft_consensus.cc:348] T b638038ba33c4697ba022eed3e8bbd64 P b0580d290b47438a9c7cace92e848865 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58c281726a7d46c2b6c754c8a4fe5bf4" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 40235 } } peers { permanent_uuid: "19e79fbaeb2f4d8f9e4cb95bc73df894" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 43723 } } peers { permanent_uuid: "b0580d290b47438a9c7cace92e848865" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 35443 } }
I20260924 13:47:17.028050 16170 raft_consensus.cc:374] T b638038ba33c4697ba022eed3e8bbd64 P b0580d290b47438a9c7cace92e848865 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 13:47:17.028071 16170 raft_consensus.cc:729] T b638038ba33c4697ba022eed3e8bbd64 P b0580d290b47438a9c7cace92e848865 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b0580d290b47438a9c7cace92e848865, State: Initialized, Role: FOLLOWER
I20260924 13:47:17.028113 16170 consensus_queue.cc:260] T b638038ba33c4697ba022eed3e8bbd64 P b0580d290b47438a9c7cace92e848865 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58c281726a7d46c2b6c754c8a4fe5bf4" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 40235 } } peers { permanent_uuid: "19e79fbaeb2f4d8f9e4cb95bc73df894" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 43723 } } peers { permanent_uuid: "b0580d290b47438a9c7cace92e848865" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 35443 } }
I20260924 13:47:17.028209 16170 ts_tablet_manager.cc:1428] T b638038ba33c4697ba022eed3e8bbd64 P b0580d290b47438a9c7cace92e848865: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260924 13:47:17.028286 16176 tablet_replica.cc:439] T b638038ba33c4697ba022eed3e8bbd64 P b0580d290b47438a9c7cace92e848865: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58c281726a7d46c2b6c754c8a4fe5bf4" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 40235 } } peers { permanent_uuid: "19e79fbaeb2f4d8f9e4cb95bc73df894" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 43723 } } peers { permanent_uuid: "b0580d290b47438a9c7cace92e848865" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 35443 } } }
I20260924 13:47:17.028375 16176 tablet_replica.cc:442] T b638038ba33c4697ba022eed3e8bbd64 P b0580d290b47438a9c7cace92e848865: This TxnStatusTablet replica's current role is: FOLLOWER
I20260924 13:47:17.028548 16171 tablet_bootstrap.cc:492] T b638038ba33c4697ba022eed3e8bbd64 P 19e79fbaeb2f4d8f9e4cb95bc73df894: Bootstrap starting.
I20260924 13:47:17.028990 16171 tablet_bootstrap.cc:654] T b638038ba33c4697ba022eed3e8bbd64 P 19e79fbaeb2f4d8f9e4cb95bc73df894: Neither blocks nor log segments found. Creating new log.
I20260924 13:47:17.029030 16172 tablet_bootstrap.cc:492] T b638038ba33c4697ba022eed3e8bbd64 P 58c281726a7d46c2b6c754c8a4fe5bf4: No bootstrap required, opened a new log
I20260924 13:47:17.029072 16172 ts_tablet_manager.cc:1397] T b638038ba33c4697ba022eed3e8bbd64 P 58c281726a7d46c2b6c754c8a4fe5bf4: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260924 13:47:17.029421 16172 raft_consensus.cc:348] T b638038ba33c4697ba022eed3e8bbd64 P 58c281726a7d46c2b6c754c8a4fe5bf4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58c281726a7d46c2b6c754c8a4fe5bf4" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 40235 } } peers { permanent_uuid: "19e79fbaeb2f4d8f9e4cb95bc73df894" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 43723 } } peers { permanent_uuid: "b0580d290b47438a9c7cace92e848865" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 35443 } }
I20260924 13:47:17.029501 16172 raft_consensus.cc:374] T b638038ba33c4697ba022eed3e8bbd64 P 58c281726a7d46c2b6c754c8a4fe5bf4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 13:47:17.029529 16172 raft_consensus.cc:729] T b638038ba33c4697ba022eed3e8bbd64 P 58c281726a7d46c2b6c754c8a4fe5bf4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 58c281726a7d46c2b6c754c8a4fe5bf4, State: Initialized, Role: FOLLOWER
I20260924 13:47:17.029582 16172 consensus_queue.cc:260] T b638038ba33c4697ba022eed3e8bbd64 P 58c281726a7d46c2b6c754c8a4fe5bf4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58c281726a7d46c2b6c754c8a4fe5bf4" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 40235 } } peers { permanent_uuid: "19e79fbaeb2f4d8f9e4cb95bc73df894" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 43723 } } peers { permanent_uuid: "b0580d290b47438a9c7cace92e848865" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 35443 } }
I20260924 13:47:17.029619 16171 tablet_bootstrap.cc:492] T b638038ba33c4697ba022eed3e8bbd64 P 19e79fbaeb2f4d8f9e4cb95bc73df894: No bootstrap required, opened a new log
I20260924 13:47:17.029659 16171 ts_tablet_manager.cc:1397] T b638038ba33c4697ba022eed3e8bbd64 P 19e79fbaeb2f4d8f9e4cb95bc73df894: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260924 13:47:17.029675 16172 ts_tablet_manager.cc:1428] T b638038ba33c4697ba022eed3e8bbd64 P 58c281726a7d46c2b6c754c8a4fe5bf4: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260924 13:47:17.029821 16178 tablet_replica.cc:439] T b638038ba33c4697ba022eed3e8bbd64 P 58c281726a7d46c2b6c754c8a4fe5bf4: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58c281726a7d46c2b6c754c8a4fe5bf4" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 40235 } } peers { permanent_uuid: "19e79fbaeb2f4d8f9e4cb95bc73df894" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 43723 } } peers { permanent_uuid: "b0580d290b47438a9c7cace92e848865" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 35443 } } }
I20260924 13:47:17.029811 16171 raft_consensus.cc:348] T b638038ba33c4697ba022eed3e8bbd64 P 19e79fbaeb2f4d8f9e4cb95bc73df894 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58c281726a7d46c2b6c754c8a4fe5bf4" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 40235 } } peers { permanent_uuid: "19e79fbaeb2f4d8f9e4cb95bc73df894" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 43723 } } peers { permanent_uuid: "b0580d290b47438a9c7cace92e848865" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 35443 } }
I20260924 13:47:17.029906 16171 raft_consensus.cc:374] T b638038ba33c4697ba022eed3e8bbd64 P 19e79fbaeb2f4d8f9e4cb95bc73df894 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 13:47:17.029908 16178 tablet_replica.cc:442] T b638038ba33c4697ba022eed3e8bbd64 P 58c281726a7d46c2b6c754c8a4fe5bf4: This TxnStatusTablet replica's current role is: FOLLOWER
I20260924 13:47:17.029927 16171 raft_consensus.cc:729] T b638038ba33c4697ba022eed3e8bbd64 P 19e79fbaeb2f4d8f9e4cb95bc73df894 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 19e79fbaeb2f4d8f9e4cb95bc73df894, State: Initialized, Role: FOLLOWER
I20260924 13:47:17.029973 16171 consensus_queue.cc:260] T b638038ba33c4697ba022eed3e8bbd64 P 19e79fbaeb2f4d8f9e4cb95bc73df894 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58c281726a7d46c2b6c754c8a4fe5bf4" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 40235 } } peers { permanent_uuid: "19e79fbaeb2f4d8f9e4cb95bc73df894" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 43723 } } peers { permanent_uuid: "b0580d290b47438a9c7cace92e848865" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 35443 } }
I20260924 13:47:17.030061 16171 ts_tablet_manager.cc:1428] T b638038ba33c4697ba022eed3e8bbd64 P 19e79fbaeb2f4d8f9e4cb95bc73df894: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260924 13:47:17.030133 16177 tablet_replica.cc:439] T b638038ba33c4697ba022eed3e8bbd64 P 19e79fbaeb2f4d8f9e4cb95bc73df894: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58c281726a7d46c2b6c754c8a4fe5bf4" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 40235 } } peers { permanent_uuid: "19e79fbaeb2f4d8f9e4cb95bc73df894" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 43723 } } peers { permanent_uuid: "b0580d290b47438a9c7cace92e848865" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 35443 } } }
I20260924 13:47:17.030201 16177 tablet_replica.cc:442] T b638038ba33c4697ba022eed3e8bbd64 P 19e79fbaeb2f4d8f9e4cb95bc73df894: This TxnStatusTablet replica's current role is: FOLLOWER
W20260924 13:47:17.109562 16148 tablet.cc:2369] T 31b03e63e4b440b5b6836b0cca1a7f1a P b0580d290b47438a9c7cace92e848865: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260924 13:47:17.135524 16177 raft_consensus.cc:482] T b638038ba33c4697ba022eed3e8bbd64 P 19e79fbaeb2f4d8f9e4cb95bc73df894 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 13:47:17.135628 16177 raft_consensus.cc:504] T b638038ba33c4697ba022eed3e8bbd64 P 19e79fbaeb2f4d8f9e4cb95bc73df894 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58c281726a7d46c2b6c754c8a4fe5bf4" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 40235 } } peers { permanent_uuid: "19e79fbaeb2f4d8f9e4cb95bc73df894" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 43723 } } peers { permanent_uuid: "b0580d290b47438a9c7cace92e848865" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 35443 } }
I20260924 13:47:17.135779 16177 leader_election.cc:288] T b638038ba33c4697ba022eed3e8bbd64 P 19e79fbaeb2f4d8f9e4cb95bc73df894 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 58c281726a7d46c2b6c754c8a4fe5bf4 (127.13.149.193:40235), b0580d290b47438a9c7cace92e848865 (127.13.149.195:35443)
I20260924 13:47:17.136041 15817 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b638038ba33c4697ba022eed3e8bbd64" candidate_uuid: "19e79fbaeb2f4d8f9e4cb95bc73df894" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58c281726a7d46c2b6c754c8a4fe5bf4" is_pre_election: true
I20260924 13:47:17.136066 16101 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b638038ba33c4697ba022eed3e8bbd64" candidate_uuid: "19e79fbaeb2f4d8f9e4cb95bc73df894" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b0580d290b47438a9c7cace92e848865" is_pre_election: true
I20260924 13:47:17.136153 16101 raft_consensus.cc:2454] T b638038ba33c4697ba022eed3e8bbd64 P b0580d290b47438a9c7cace92e848865 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19e79fbaeb2f4d8f9e4cb95bc73df894 in term 0.
I20260924 13:47:17.136153 15817 raft_consensus.cc:2454] T b638038ba33c4697ba022eed3e8bbd64 P 58c281726a7d46c2b6c754c8a4fe5bf4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19e79fbaeb2f4d8f9e4cb95bc73df894 in term 0.
I20260924 13:47:17.136309 15890 leader_election.cc:302] T b638038ba33c4697ba022eed3e8bbd64 P 19e79fbaeb2f4d8f9e4cb95bc73df894 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19e79fbaeb2f4d8f9e4cb95bc73df894, b0580d290b47438a9c7cace92e848865; no voters:
I20260924 13:47:17.136416 16177 raft_consensus.cc:2781] T b638038ba33c4697ba022eed3e8bbd64 P 19e79fbaeb2f4d8f9e4cb95bc73df894 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 13:47:17.136476 16177 raft_consensus.cc:482] T b638038ba33c4697ba022eed3e8bbd64 P 19e79fbaeb2f4d8f9e4cb95bc73df894 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 13:47:17.136497 16177 raft_consensus.cc:3037] T b638038ba33c4697ba022eed3e8bbd64 P 19e79fbaeb2f4d8f9e4cb95bc73df894 [term 0 FOLLOWER]: Advancing to term 1
I20260924 13:47:17.137244 16177 raft_consensus.cc:504] T b638038ba33c4697ba022eed3e8bbd64 P 19e79fbaeb2f4d8f9e4cb95bc73df894 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58c281726a7d46c2b6c754c8a4fe5bf4" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 40235 } } peers { permanent_uuid: "19e79fbaeb2f4d8f9e4cb95bc73df894" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 43723 } } peers { permanent_uuid: "b0580d290b47438a9c7cace92e848865" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 35443 } }
I20260924 13:47:17.137370 16177 leader_election.cc:288] T b638038ba33c4697ba022eed3e8bbd64 P 19e79fbaeb2f4d8f9e4cb95bc73df894 [CANDIDATE]: Term 1 election: Requested vote from peers 58c281726a7d46c2b6c754c8a4fe5bf4 (127.13.149.193:40235), b0580d290b47438a9c7cace92e848865 (127.13.149.195:35443)
I20260924 13:47:17.137636 15817 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b638038ba33c4697ba022eed3e8bbd64" candidate_uuid: "19e79fbaeb2f4d8f9e4cb95bc73df894" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58c281726a7d46c2b6c754c8a4fe5bf4"
I20260924 13:47:17.137667 16101 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b638038ba33c4697ba022eed3e8bbd64" candidate_uuid: "19e79fbaeb2f4d8f9e4cb95bc73df894" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b0580d290b47438a9c7cace92e848865"
I20260924 13:47:17.137739 15817 raft_consensus.cc:3037] T b638038ba33c4697ba022eed3e8bbd64 P 58c281726a7d46c2b6c754c8a4fe5bf4 [term 0 FOLLOWER]: Advancing to term 1
I20260924 13:47:17.137738 16101 raft_consensus.cc:3037] T b638038ba33c4697ba022eed3e8bbd64 P b0580d290b47438a9c7cace92e848865 [term 0 FOLLOWER]: Advancing to term 1
I20260924 13:47:17.138521 15817 raft_consensus.cc:2454] T b638038ba33c4697ba022eed3e8bbd64 P 58c281726a7d46c2b6c754c8a4fe5bf4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19e79fbaeb2f4d8f9e4cb95bc73df894 in term 1.
I20260924 13:47:17.138521 16101 raft_consensus.cc:2454] T b638038ba33c4697ba022eed3e8bbd64 P b0580d290b47438a9c7cace92e848865 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19e79fbaeb2f4d8f9e4cb95bc73df894 in term 1.
I20260924 13:47:17.138787 15890 leader_election.cc:302] T b638038ba33c4697ba022eed3e8bbd64 P 19e79fbaeb2f4d8f9e4cb95bc73df894 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19e79fbaeb2f4d8f9e4cb95bc73df894, 58c281726a7d46c2b6c754c8a4fe5bf4; no voters:
I20260924 13:47:17.138890 16177 raft_consensus.cc:2781] T b638038ba33c4697ba022eed3e8bbd64 P 19e79fbaeb2f4d8f9e4cb95bc73df894 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 13:47:17.138962 16177 raft_consensus.cc:686] T b638038ba33c4697ba022eed3e8bbd64 P 19e79fbaeb2f4d8f9e4cb95bc73df894 [term 1 LEADER]: Becoming Leader. State: Replica: 19e79fbaeb2f4d8f9e4cb95bc73df894, State: Running, Role: LEADER
I20260924 13:47:17.139016 16177 consensus_queue.cc:237] T b638038ba33c4697ba022eed3e8bbd64 P 19e79fbaeb2f4d8f9e4cb95bc73df894 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58c281726a7d46c2b6c754c8a4fe5bf4" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 40235 } } peers { permanent_uuid: "19e79fbaeb2f4d8f9e4cb95bc73df894" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 43723 } } peers { permanent_uuid: "b0580d290b47438a9c7cace92e848865" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 35443 } }
I20260924 13:47:17.139322 16180 tablet_replica.cc:439] T b638038ba33c4697ba022eed3e8bbd64 P 19e79fbaeb2f4d8f9e4cb95bc73df894: TxnStatusTablet state changed. Reason: New leader 19e79fbaeb2f4d8f9e4cb95bc73df894. Latest consensus state: current_term: 1 leader_uuid: "19e79fbaeb2f4d8f9e4cb95bc73df894" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58c281726a7d46c2b6c754c8a4fe5bf4" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 40235 } } peers { permanent_uuid: "19e79fbaeb2f4d8f9e4cb95bc73df894" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 43723 } } peers { permanent_uuid: "b0580d290b47438a9c7cace92e848865" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 35443 } } }
I20260924 13:47:17.139405 16180 tablet_replica.cc:442] T b638038ba33c4697ba022eed3e8bbd64 P 19e79fbaeb2f4d8f9e4cb95bc73df894: This TxnStatusTablet replica's current role is: LEADER
I20260924 13:47:17.139670 15657 catalog_manager.cc:5495] T b638038ba33c4697ba022eed3e8bbd64 P 19e79fbaeb2f4d8f9e4cb95bc73df894 reported cstate change: term changed from 0 to 1, leader changed from <none> to 19e79fbaeb2f4d8f9e4cb95bc73df894 (127.13.149.194). New cstate: current_term: 1 leader_uuid: "19e79fbaeb2f4d8f9e4cb95bc73df894" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58c281726a7d46c2b6c754c8a4fe5bf4" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 40235 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "19e79fbaeb2f4d8f9e4cb95bc73df894" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 43723 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b0580d290b47438a9c7cace92e848865" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 35443 } health_report { overall_health: UNKNOWN } } }
I20260924 13:47:17.139997 16192 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260924 13:47:17.140136 16192 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260924 13:47:17.147964 16187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:17.145113 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.194:43723 (local address 127.0.0.1:53162)
0924 13:47:17.145198 (+ 85us) negotiation.cc:107] Waiting for socket to connect
0924 13:47:17.145212 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0924 13:47:17.145262 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:47:17.145446 (+ 184us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:47:17.145447 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:47:17.145448 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0924 13:47:17.145502 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:47:17.145505 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:47:17.146211 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:47:17.146212 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:47:17.146858 (+ 646us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:47:17.146860 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:47:17.147394 (+ 534us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:47:17.147395 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:47:17.147445 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:47:17.147467 (+ 22us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0924 13:47:17.147786 (+ 319us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0924 13:47:17.147799 (+ 13us) client_negotiation.cc:769] Sending connection context
0924 13:47:17.147848 (+ 49us) client_negotiation.cc:240] Negotiation successful
0924 13:47:17.147899 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31}
I20260924 13:47:17.147964 16167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:17.145182 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53162 (local address 127.13.149.194:43723)
0924 13:47:17.145244 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0924 13:47:17.145246 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:47:17.145256 (+ 10us) server_negotiation.cc:407] Connection header received
0924 13:47:17.145337 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:47:17.145338 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:47:17.145359 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:47:17.145432 (+ 73us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 13:47:17.145573 (+ 141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:17.146090 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:17.146941 (+ 851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:17.147314 (+ 373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:17.147397 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:47:17.147552 (+ 155us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 13:47:17.147687 (+ 135us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 13:47:17.147775 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0924 13:47:17.147853 (+ 78us) server_negotiation.cc:299] Negotiation successful
0924 13:47:17.147877 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260924 13:47:17.149936 16101 raft_consensus.cc:1261] T b638038ba33c4697ba022eed3e8bbd64 P b0580d290b47438a9c7cace92e848865 [term 1 FOLLOWER]: Refusing update from remote peer 19e79fbaeb2f4d8f9e4cb95bc73df894: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 13:47:17.149936 15817 raft_consensus.cc:1261] T b638038ba33c4697ba022eed3e8bbd64 P 58c281726a7d46c2b6c754c8a4fe5bf4 [term 1 FOLLOWER]: Refusing update from remote peer 19e79fbaeb2f4d8f9e4cb95bc73df894: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 13:47:17.150269 16177 consensus_queue.cc:1035] T b638038ba33c4697ba022eed3e8bbd64 P 19e79fbaeb2f4d8f9e4cb95bc73df894 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58c281726a7d46c2b6c754c8a4fe5bf4" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 40235 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 13:47:17.150404 16180 consensus_queue.cc:1035] T b638038ba33c4697ba022eed3e8bbd64 P 19e79fbaeb2f4d8f9e4cb95bc73df894 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b0580d290b47438a9c7cace92e848865" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 35443 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 13:47:17.151561 16178 tablet_replica.cc:439] T b638038ba33c4697ba022eed3e8bbd64 P 58c281726a7d46c2b6c754c8a4fe5bf4: TxnStatusTablet state changed. Reason: New leader 19e79fbaeb2f4d8f9e4cb95bc73df894. Latest consensus state: current_term: 1 leader_uuid: "19e79fbaeb2f4d8f9e4cb95bc73df894" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58c281726a7d46c2b6c754c8a4fe5bf4" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 40235 } } peers { permanent_uuid: "19e79fbaeb2f4d8f9e4cb95bc73df894" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 43723 } } peers { permanent_uuid: "b0580d290b47438a9c7cace92e848865" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 35443 } } }
I20260924 13:47:17.151657 16178 tablet_replica.cc:442] T b638038ba33c4697ba022eed3e8bbd64 P 58c281726a7d46c2b6c754c8a4fe5bf4: This TxnStatusTablet replica's current role is: FOLLOWER
I20260924 13:47:17.151590 16176 tablet_replica.cc:439] T b638038ba33c4697ba022eed3e8bbd64 P b0580d290b47438a9c7cace92e848865: TxnStatusTablet state changed. Reason: New leader 19e79fbaeb2f4d8f9e4cb95bc73df894. Latest consensus state: current_term: 1 leader_uuid: "19e79fbaeb2f4d8f9e4cb95bc73df894" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58c281726a7d46c2b6c754c8a4fe5bf4" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 40235 } } peers { permanent_uuid: "19e79fbaeb2f4d8f9e4cb95bc73df894" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 43723 } } peers { permanent_uuid: "b0580d290b47438a9c7cace92e848865" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 35443 } } }
I20260924 13:47:17.151690 16176 tablet_replica.cc:442] T b638038ba33c4697ba022eed3e8bbd64 P b0580d290b47438a9c7cace92e848865: This TxnStatusTablet replica's current role is: FOLLOWER
I20260924 13:47:17.152609 16177 tablet_replica.cc:439] T b638038ba33c4697ba022eed3e8bbd64 P 19e79fbaeb2f4d8f9e4cb95bc73df894: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "19e79fbaeb2f4d8f9e4cb95bc73df894" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58c281726a7d46c2b6c754c8a4fe5bf4" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 40235 } } peers { permanent_uuid: "19e79fbaeb2f4d8f9e4cb95bc73df894" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 43723 } } peers { permanent_uuid: "b0580d290b47438a9c7cace92e848865" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 35443 } } }
I20260924 13:47:17.152704 16177 tablet_replica.cc:442] T b638038ba33c4697ba022eed3e8bbd64 P 19e79fbaeb2f4d8f9e4cb95bc73df894: This TxnStatusTablet replica's current role is: LEADER
I20260924 13:47:17.153023 16180 tablet_replica.cc:439] T b638038ba33c4697ba022eed3e8bbd64 P 19e79fbaeb2f4d8f9e4cb95bc73df894: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "19e79fbaeb2f4d8f9e4cb95bc73df894" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58c281726a7d46c2b6c754c8a4fe5bf4" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 40235 } } peers { permanent_uuid: "19e79fbaeb2f4d8f9e4cb95bc73df894" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 43723 } } peers { permanent_uuid: "b0580d290b47438a9c7cace92e848865" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 35443 } } }
I20260924 13:47:17.153105 16180 tablet_replica.cc:442] T b638038ba33c4697ba022eed3e8bbd64 P 19e79fbaeb2f4d8f9e4cb95bc73df894: This TxnStatusTablet replica's current role is: LEADER
I20260924 13:47:17.153347 16196 mvcc.cc:205] Tried to move back new op lower bound from 7332895281763414016 to 7332895281721753600. Current Snapshot: MvccSnapshot[applied={T|T < 7332895281763414016}]
I20260924 13:47:17.153338 16178 tablet_replica.cc:439] T b638038ba33c4697ba022eed3e8bbd64 P 58c281726a7d46c2b6c754c8a4fe5bf4: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "19e79fbaeb2f4d8f9e4cb95bc73df894" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58c281726a7d46c2b6c754c8a4fe5bf4" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 40235 } } peers { permanent_uuid: "19e79fbaeb2f4d8f9e4cb95bc73df894" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 43723 } } peers { permanent_uuid: "b0580d290b47438a9c7cace92e848865" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 35443 } } }
I20260924 13:47:17.153434 16178 tablet_replica.cc:442] T b638038ba33c4697ba022eed3e8bbd64 P 58c281726a7d46c2b6c754c8a4fe5bf4: This TxnStatusTablet replica's current role is: FOLLOWER
I20260924 13:47:17.153622 16176 tablet_replica.cc:439] T b638038ba33c4697ba022eed3e8bbd64 P b0580d290b47438a9c7cace92e848865: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "19e79fbaeb2f4d8f9e4cb95bc73df894" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58c281726a7d46c2b6c754c8a4fe5bf4" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 40235 } } peers { permanent_uuid: "19e79fbaeb2f4d8f9e4cb95bc73df894" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 43723 } } peers { permanent_uuid: "b0580d290b47438a9c7cace92e848865" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 35443 } } }
I20260924 13:47:17.153712 16176 tablet_replica.cc:442] T b638038ba33c4697ba022eed3e8bbd64 P b0580d290b47438a9c7cace92e848865: This TxnStatusTablet replica's current role is: FOLLOWER
I20260924 13:47:17.155025 16198 mvcc.cc:205] Tried to move back new op lower bound from 7332895281763414016 to 7332895281721753600. Current Snapshot: MvccSnapshot[applied={T|T < 7332895281763414016}]
I20260924 13:47:17.155164 16197 mvcc.cc:205] Tried to move back new op lower bound from 7332895281763414016 to 7332895281721753600. Current Snapshot: MvccSnapshot[applied={T|T < 7332895281763414016 or (T in {7332895281763414016})}]
I20260924 13:47:17.158717 16167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:17.156097 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53176 (local address 127.13.149.194:43723)
0924 13:47:17.156195 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0924 13:47:17.156197 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:47:17.156242 (+ 45us) server_negotiation.cc:407] Connection header received
0924 13:47:17.156463 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:47:17.156464 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:47:17.156477 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:47:17.156524 (+ 47us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 13:47:17.156660 (+ 136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:17.157318 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:17.158082 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:17.158293 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:17.158348 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:47:17.158483 (+ 135us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 13:47:17.158516 (+ 33us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 13:47:17.158555 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0924 13:47:17.158611 (+ 56us) server_negotiation.cc:299] Negotiation successful
0924 13:47:17.158633 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260924 13:47:17.164525 16167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:17.161779 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.149.194:51699 (local address 127.13.149.194:43723)
0924 13:47:17.161811 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0924 13:47:17.161813 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:47:17.161821 (+ 8us) server_negotiation.cc:407] Connection header received
0924 13:47:17.161892 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:47:17.161893 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:47:17.161904 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:47:17.161955 (+ 51us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 13:47:17.162081 (+ 126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:17.162837 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:17.163652 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:17.163961 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:17.164044 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:47:17.164191 (+ 147us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 13:47:17.164226 (+ 35us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 13:47:17.164277 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0924 13:47:17.164379 (+ 102us) server_negotiation.cc:299] Negotiation successful
0924 13:47:17.164400 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":16}
I20260924 13:47:17.164525 16012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:17.161713 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.194:43723 (local address 127.13.149.194:51699)
0924 13:47:17.161766 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0924 13:47:17.161776 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 13:47:17.161818 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:47:17.161962 (+ 144us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:47:17.161963 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:47:17.161964 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0924 13:47:17.162027 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:47:17.162028 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:47:17.162948 (+ 920us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:47:17.162949 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:47:17.163569 (+ 620us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:47:17.163571 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:47:17.164033 (+ 462us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:47:17.164033 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:47:17.164082 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:47:17.164101 (+ 19us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0924 13:47:17.164308 (+ 207us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0924 13:47:17.164311 (+ 3us) client_negotiation.cc:769] Sending connection context
0924 13:47:17.164360 (+ 49us) client_negotiation.cc:240] Negotiation successful
0924 13:47:17.164406 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":20}
I20260924 13:47:17.167021 15817 raft_consensus.cc:1261] T 31b03e63e4b440b5b6836b0cca1a7f1a P 58c281726a7d46c2b6c754c8a4fe5bf4 [term 1 FOLLOWER]: Refusing update from remote peer 19e79fbaeb2f4d8f9e4cb95bc73df894: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 13:47:17.167018 16101 raft_consensus.cc:1261] T 31b03e63e4b440b5b6836b0cca1a7f1a P b0580d290b47438a9c7cace92e848865 [term 1 FOLLOWER]: Refusing update from remote peer 19e79fbaeb2f4d8f9e4cb95bc73df894: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 13:47:17.167259 16177 consensus_queue.cc:1035] T 31b03e63e4b440b5b6836b0cca1a7f1a P 19e79fbaeb2f4d8f9e4cb95bc73df894 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b0580d290b47438a9c7cace92e848865" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 35443 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 13:47:17.167392 16207 consensus_queue.cc:1035] T 31b03e63e4b440b5b6836b0cca1a7f1a P 19e79fbaeb2f4d8f9e4cb95bc73df894 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58c281726a7d46c2b6c754c8a4fe5bf4" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 40235 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 13:47:17.179772 15657 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42852:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 13:47:17.179908 15657 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42852:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 13:47:17.180682 15657 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4dbf8e8088ce4e99ba854b50554eeb6a: Sending DeleteTablet for 3 replicas of tablet 31b03e63e4b440b5b6836b0cca1a7f1a
I20260924 13:47:17.181121 16081 tablet_service.cc:1514] Processing DeleteTablet for tablet 31b03e63e4b440b5b6836b0cca1a7f1a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 13:47:17 UTC) from {username='kudu', principal='kudu/127.13.149.254@KRBTEST.COM'} at 127.0.0.1:46412
I20260924 13:47:17.181131 15797 tablet_service.cc:1514] Processing DeleteTablet for tablet 31b03e63e4b440b5b6836b0cca1a7f1a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 13:47:17 UTC) from {username='kudu', principal='kudu/127.13.149.254@KRBTEST.COM'} at 127.0.0.1:46830
I20260924 13:47:17.181167 15938 tablet_service.cc:1514] Processing DeleteTablet for tablet 31b03e63e4b440b5b6836b0cca1a7f1a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 13:47:17 UTC) from {username='kudu', principal='kudu/127.13.149.254@KRBTEST.COM'} at 127.0.0.1:53148
I20260924 13:47:17.181524 16212 tablet_replica.cc:330] T 31b03e63e4b440b5b6836b0cca1a7f1a P 19e79fbaeb2f4d8f9e4cb95bc73df894: stopping tablet replica
I20260924 13:47:17.181491 15657 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42852:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 13:47:17.181643 15657 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 13:47:17.181703 16212 raft_consensus.cc:2229] T 31b03e63e4b440b5b6836b0cca1a7f1a P 19e79fbaeb2f4d8f9e4cb95bc73df894 [term 1 LEADER]: Raft consensus shutting down.
I20260924 13:47:17.181811 16212 raft_consensus.cc:2258] T 31b03e63e4b440b5b6836b0cca1a7f1a P 19e79fbaeb2f4d8f9e4cb95bc73df894 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 13:47:17.181952 16213 tablet_replica.cc:330] T 31b03e63e4b440b5b6836b0cca1a7f1a P 58c281726a7d46c2b6c754c8a4fe5bf4: stopping tablet replica
I20260924 13:47:17.182060 16213 raft_consensus.cc:2229] T 31b03e63e4b440b5b6836b0cca1a7f1a P 58c281726a7d46c2b6c754c8a4fe5bf4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 13:47:17.182114 16213 raft_consensus.cc:2258] T 31b03e63e4b440b5b6836b0cca1a7f1a P 58c281726a7d46c2b6c754c8a4fe5bf4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 13:47:17.182858 16212 ts_tablet_manager.cc:1905] T 31b03e63e4b440b5b6836b0cca1a7f1a P 19e79fbaeb2f4d8f9e4cb95bc73df894: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 13:47:17.183054 15938 tablet_service.cc:1467] Processing CreateTablet for tablet bb1e9376203c4e43bcaa2492c287d7af (DEFAULT_TABLE table=test-table [id=b58fc0aa6a5c4d19be6fdd6bc6b0ff44]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 13:47:17.183055 16081 tablet_service.cc:1467] Processing CreateTablet for tablet bb1e9376203c4e43bcaa2492c287d7af (DEFAULT_TABLE table=test-table [id=b58fc0aa6a5c4d19be6fdd6bc6b0ff44]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 13:47:17.183061 15797 tablet_service.cc:1467] Processing CreateTablet for tablet bb1e9376203c4e43bcaa2492c287d7af (DEFAULT_TABLE table=test-table [id=b58fc0aa6a5c4d19be6fdd6bc6b0ff44]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 13:47:17.183168 15797 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bb1e9376203c4e43bcaa2492c287d7af. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 13:47:17.183168 16081 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bb1e9376203c4e43bcaa2492c287d7af. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 13:47:17.183168 15938 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bb1e9376203c4e43bcaa2492c287d7af. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 13:47:17.184470 16170 tablet_bootstrap.cc:492] T bb1e9376203c4e43bcaa2492c287d7af P b0580d290b47438a9c7cace92e848865: Bootstrap starting.
I20260924 13:47:17.184476 16172 tablet_bootstrap.cc:492] T bb1e9376203c4e43bcaa2492c287d7af P 58c281726a7d46c2b6c754c8a4fe5bf4: Bootstrap starting.
I20260924 13:47:17.184525 16171 tablet_bootstrap.cc:492] T bb1e9376203c4e43bcaa2492c287d7af P 19e79fbaeb2f4d8f9e4cb95bc73df894: Bootstrap starting.
I20260924 13:47:17.184764 16213 ts_tablet_manager.cc:1905] T 31b03e63e4b440b5b6836b0cca1a7f1a P 58c281726a7d46c2b6c754c8a4fe5bf4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 13:47:17.184867 16172 tablet_bootstrap.cc:654] T bb1e9376203c4e43bcaa2492c287d7af P 58c281726a7d46c2b6c754c8a4fe5bf4: Neither blocks nor log segments found. Creating new log.
I20260924 13:47:17.184911 16170 tablet_bootstrap.cc:654] T bb1e9376203c4e43bcaa2492c287d7af P b0580d290b47438a9c7cace92e848865: Neither blocks nor log segments found. Creating new log.
I20260924 13:47:17.184911 16171 tablet_bootstrap.cc:654] T bb1e9376203c4e43bcaa2492c287d7af P 19e79fbaeb2f4d8f9e4cb95bc73df894: Neither blocks nor log segments found. Creating new log.
I20260924 13:47:17.185719 16170 tablet_bootstrap.cc:492] T bb1e9376203c4e43bcaa2492c287d7af P b0580d290b47438a9c7cace92e848865: No bootstrap required, opened a new log
I20260924 13:47:17.185766 16170 ts_tablet_manager.cc:1397] T bb1e9376203c4e43bcaa2492c287d7af P b0580d290b47438a9c7cace92e848865: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260924 13:47:17.185899 16170 raft_consensus.cc:348] T bb1e9376203c4e43bcaa2492c287d7af P b0580d290b47438a9c7cace92e848865 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58c281726a7d46c2b6c754c8a4fe5bf4" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 40235 } } peers { permanent_uuid: "19e79fbaeb2f4d8f9e4cb95bc73df894" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 43723 } } peers { permanent_uuid: "b0580d290b47438a9c7cace92e848865" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 35443 } }
I20260924 13:47:17.185978 16170 raft_consensus.cc:374] T bb1e9376203c4e43bcaa2492c287d7af P b0580d290b47438a9c7cace92e848865 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 13:47:17.185998 16170 raft_consensus.cc:729] T bb1e9376203c4e43bcaa2492c287d7af P b0580d290b47438a9c7cace92e848865 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b0580d290b47438a9c7cace92e848865, State: Initialized, Role: FOLLOWER
I20260924 13:47:17.186048 16170 consensus_queue.cc:260] T bb1e9376203c4e43bcaa2492c287d7af P b0580d290b47438a9c7cace92e848865 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58c281726a7d46c2b6c754c8a4fe5bf4" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 40235 } } peers { permanent_uuid: "19e79fbaeb2f4d8f9e4cb95bc73df894" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 43723 } } peers { permanent_uuid: "b0580d290b47438a9c7cace92e848865" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 35443 } }
I20260924 13:47:17.186137 16170 ts_tablet_manager.cc:1428] T bb1e9376203c4e43bcaa2492c287d7af P b0580d290b47438a9c7cace92e848865: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260924 13:47:17.186203 16213 ts_tablet_manager.cc:1918] T 31b03e63e4b440b5b6836b0cca1a7f1a P 58c281726a7d46c2b6c754c8a4fe5bf4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260924 13:47:17.186278 16213 log.cc:1196] T 31b03e63e4b440b5b6836b0cca1a7f1a P 58c281726a7d46c2b6c754c8a4fe5bf4: Deleting WAL directory at /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790257629041645-13911-0/minicluster-data/ts-0/wal/wals/31b03e63e4b440b5b6836b0cca1a7f1a
I20260924 13:47:17.186520 16213 ts_tablet_manager.cc:1939] T 31b03e63e4b440b5b6836b0cca1a7f1a P 58c281726a7d46c2b6c754c8a4fe5bf4: Deleting consensus metadata
I20260924 13:47:17.186931 15644 catalog_manager.cc:4841] TS 58c281726a7d46c2b6c754c8a4fe5bf4 (127.13.149.193:40235): tablet 31b03e63e4b440b5b6836b0cca1a7f1a (table test-table [id=28d8485f7a33466db4d76b05cfe8ee5f]) successfully deleted
I20260924 13:47:17.187175 16172 tablet_bootstrap.cc:492] T bb1e9376203c4e43bcaa2492c287d7af P 58c281726a7d46c2b6c754c8a4fe5bf4: No bootstrap required, opened a new log
I20260924 13:47:17.187225 16172 ts_tablet_manager.cc:1397] T bb1e9376203c4e43bcaa2492c287d7af P 58c281726a7d46c2b6c754c8a4fe5bf4: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20260924 13:47:17.187309 16171 tablet_bootstrap.cc:492] T bb1e9376203c4e43bcaa2492c287d7af P 19e79fbaeb2f4d8f9e4cb95bc73df894: No bootstrap required, opened a new log
I20260924 13:47:17.187352 16171 ts_tablet_manager.cc:1397] T bb1e9376203c4e43bcaa2492c287d7af P 19e79fbaeb2f4d8f9e4cb95bc73df894: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20260924 13:47:17.187362 16172 raft_consensus.cc:348] T bb1e9376203c4e43bcaa2492c287d7af P 58c281726a7d46c2b6c754c8a4fe5bf4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58c281726a7d46c2b6c754c8a4fe5bf4" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 40235 } } peers { permanent_uuid: "19e79fbaeb2f4d8f9e4cb95bc73df894" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 43723 } } peers { permanent_uuid: "b0580d290b47438a9c7cace92e848865" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 35443 } }
I20260924 13:47:17.187425 16172 raft_consensus.cc:374] T bb1e9376203c4e43bcaa2492c287d7af P 58c281726a7d46c2b6c754c8a4fe5bf4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 13:47:17.187443 16172 raft_consensus.cc:729] T bb1e9376203c4e43bcaa2492c287d7af P 58c281726a7d46c2b6c754c8a4fe5bf4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 58c281726a7d46c2b6c754c8a4fe5bf4, State: Initialized, Role: FOLLOWER
I20260924 13:47:17.187482 16172 consensus_queue.cc:260] T bb1e9376203c4e43bcaa2492c287d7af P 58c281726a7d46c2b6c754c8a4fe5bf4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58c281726a7d46c2b6c754c8a4fe5bf4" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 40235 } } peers { permanent_uuid: "19e79fbaeb2f4d8f9e4cb95bc73df894" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 43723 } } peers { permanent_uuid: "b0580d290b47438a9c7cace92e848865" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 35443 } }
I20260924 13:47:17.187497 16171 raft_consensus.cc:348] T bb1e9376203c4e43bcaa2492c287d7af P 19e79fbaeb2f4d8f9e4cb95bc73df894 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58c281726a7d46c2b6c754c8a4fe5bf4" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 40235 } } peers { permanent_uuid: "19e79fbaeb2f4d8f9e4cb95bc73df894" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 43723 } } peers { permanent_uuid: "b0580d290b47438a9c7cace92e848865" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 35443 } }
I20260924 13:47:17.187556 16172 ts_tablet_manager.cc:1428] T bb1e9376203c4e43bcaa2492c287d7af P 58c281726a7d46c2b6c754c8a4fe5bf4: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260924 13:47:17.187561 16171 raft_consensus.cc:374] T bb1e9376203c4e43bcaa2492c287d7af P 19e79fbaeb2f4d8f9e4cb95bc73df894 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 13:47:17.187580 16171 raft_consensus.cc:729] T bb1e9376203c4e43bcaa2492c287d7af P 19e79fbaeb2f4d8f9e4cb95bc73df894 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 19e79fbaeb2f4d8f9e4cb95bc73df894, State: Initialized, Role: FOLLOWER
I20260924 13:47:17.187618 16171 consensus_queue.cc:260] T bb1e9376203c4e43bcaa2492c287d7af P 19e79fbaeb2f4d8f9e4cb95bc73df894 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58c281726a7d46c2b6c754c8a4fe5bf4" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 40235 } } peers { permanent_uuid: "19e79fbaeb2f4d8f9e4cb95bc73df894" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 43723 } } peers { permanent_uuid: "b0580d290b47438a9c7cace92e848865" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 35443 } }
I20260924 13:47:17.187700 16171 ts_tablet_manager.cc:1428] T bb1e9376203c4e43bcaa2492c287d7af P 19e79fbaeb2f4d8f9e4cb95bc73df894: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260924 13:47:17.189074 16212 ts_tablet_manager.cc:1918] T 31b03e63e4b440b5b6836b0cca1a7f1a P 19e79fbaeb2f4d8f9e4cb95bc73df894: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260924 13:47:17.189164 16212 log.cc:1196] T 31b03e63e4b440b5b6836b0cca1a7f1a P 19e79fbaeb2f4d8f9e4cb95bc73df894: Deleting WAL directory at /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790257629041645-13911-0/minicluster-data/ts-1/wal/wals/31b03e63e4b440b5b6836b0cca1a7f1a
I20260924 13:47:17.189429 16212 ts_tablet_manager.cc:1939] T 31b03e63e4b440b5b6836b0cca1a7f1a P 19e79fbaeb2f4d8f9e4cb95bc73df894: Deleting consensus metadata
I20260924 13:47:17.189841 16211 tablet_replica.cc:330] T 31b03e63e4b440b5b6836b0cca1a7f1a P b0580d290b47438a9c7cace92e848865: stopping tablet replica
I20260924 13:47:17.189874 15643 catalog_manager.cc:4841] TS 19e79fbaeb2f4d8f9e4cb95bc73df894 (127.13.149.194:43723): tablet 31b03e63e4b440b5b6836b0cca1a7f1a (table test-table [id=28d8485f7a33466db4d76b05cfe8ee5f]) successfully deleted
I20260924 13:47:17.189975 16211 raft_consensus.cc:2229] T 31b03e63e4b440b5b6836b0cca1a7f1a P b0580d290b47438a9c7cace92e848865 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 13:47:17.190049 16211 raft_consensus.cc:2258] T 31b03e63e4b440b5b6836b0cca1a7f1a P b0580d290b47438a9c7cace92e848865 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 13:47:17.190512 16211 ts_tablet_manager.cc:1905] T 31b03e63e4b440b5b6836b0cca1a7f1a P b0580d290b47438a9c7cace92e848865: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 13:47:17.192974 16211 ts_tablet_manager.cc:1918] T 31b03e63e4b440b5b6836b0cca1a7f1a P b0580d290b47438a9c7cace92e848865: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260924 13:47:17.193063 16211 log.cc:1196] T 31b03e63e4b440b5b6836b0cca1a7f1a P b0580d290b47438a9c7cace92e848865: Deleting WAL directory at /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790257629041645-13911-0/minicluster-data/ts-2/wal/wals/31b03e63e4b440b5b6836b0cca1a7f1a
I20260924 13:47:17.193300 16211 ts_tablet_manager.cc:1939] T 31b03e63e4b440b5b6836b0cca1a7f1a P b0580d290b47438a9c7cace92e848865: Deleting consensus metadata
I20260924 13:47:17.193838 15644 catalog_manager.cc:4841] TS b0580d290b47438a9c7cace92e848865 (127.13.149.195:35443): tablet 31b03e63e4b440b5b6836b0cca1a7f1a (table test-table [id=28d8485f7a33466db4d76b05cfe8ee5f]) successfully deleted
I20260924 13:47:17.256372 16178 raft_consensus.cc:482] T bb1e9376203c4e43bcaa2492c287d7af P 58c281726a7d46c2b6c754c8a4fe5bf4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 13:47:17.256489 16178 raft_consensus.cc:504] T bb1e9376203c4e43bcaa2492c287d7af P 58c281726a7d46c2b6c754c8a4fe5bf4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58c281726a7d46c2b6c754c8a4fe5bf4" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 40235 } } peers { permanent_uuid: "19e79fbaeb2f4d8f9e4cb95bc73df894" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 43723 } } peers { permanent_uuid: "b0580d290b47438a9c7cace92e848865" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 35443 } }
I20260924 13:47:17.256843 16178 leader_election.cc:288] T bb1e9376203c4e43bcaa2492c287d7af P 58c281726a7d46c2b6c754c8a4fe5bf4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 19e79fbaeb2f4d8f9e4cb95bc73df894 (127.13.149.194:43723), b0580d290b47438a9c7cace92e848865 (127.13.149.195:35443)
I20260924 13:47:17.260301 16218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:17.257071 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.194:43723 (local address 127.13.149.193:43081)
0924 13:47:17.257259 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0924 13:47:17.257271 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0924 13:47:17.257364 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:47:17.257532 (+ 168us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:47:17.257533 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:47:17.257534 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 13:47:17.257580 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:47:17.257581 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:47:17.258507 (+ 926us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:47:17.258509 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:47:17.259475 (+ 966us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:47:17.259478 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:47:17.260014 (+ 536us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:47:17.260015 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:47:17.260088 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:47:17.260093 (+ 5us) client_negotiation.cc:769] Sending connection context
0924 13:47:17.260146 (+ 53us) client_negotiation.cc:240] Negotiation successful
0924 13:47:17.260189 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":66,"threads_started":1}
I20260924 13:47:17.260314 16167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:17.257176 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.149.193:43081 (local address 127.13.149.194:43723)
0924 13:47:17.257228 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0924 13:47:17.257230 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:47:17.257340 (+ 110us) server_negotiation.cc:407] Connection header received
0924 13:47:17.257436 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:47:17.257437 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:47:17.257446 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:47:17.257468 (+ 22us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 13:47:17.257656 (+ 188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:17.258411 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:17.259557 (+ 1146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:17.259948 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:17.260013 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:47:17.260023 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0924 13:47:17.260174 (+ 151us) server_negotiation.cc:299] Negotiation successful
0924 13:47:17.260218 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260924 13:47:17.260953 15959 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bb1e9376203c4e43bcaa2492c287d7af" candidate_uuid: "58c281726a7d46c2b6c754c8a4fe5bf4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19e79fbaeb2f4d8f9e4cb95bc73df894" is_pre_election: true
I20260924 13:47:17.261032 16169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:17.257283 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.149.193:34317 (local address 127.13.149.195:35443)
0924 13:47:17.257347 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0924 13:47:17.257349 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:47:17.257359 (+ 10us) server_negotiation.cc:407] Connection header received
0924 13:47:17.257417 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:47:17.257418 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:47:17.257427 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:47:17.257491 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 13:47:17.258060 (+ 569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:17.258683 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:17.260099 (+ 1416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:17.260637 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:17.260717 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:47:17.260727 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0924 13:47:17.260899 (+ 172us) server_negotiation.cc:299] Negotiation successful
0924 13:47:17.260971 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260924 13:47:17.261096 16217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:17.257071 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.195:35443 (local address 127.13.149.193:34317)
0924 13:47:17.257300 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0924 13:47:17.257308 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0924 13:47:17.257352 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:47:17.257502 (+ 150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:47:17.257503 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:47:17.257505 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 13:47:17.257580 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:47:17.257581 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:47:17.258825 (+ 1244us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:47:17.258826 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:47:17.259972 (+ 1146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:47:17.259975 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:47:17.260740 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:47:17.260742 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:47:17.260826 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:47:17.260830 (+ 4us) client_negotiation.cc:769] Sending connection context
0924 13:47:17.260891 (+ 61us) client_negotiation.cc:240] Negotiation successful
0924 13:47:17.260985 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":216,"thread_start_us":90,"threads_started":1}
I20260924 13:47:17.261102 15959 raft_consensus.cc:2454] T bb1e9376203c4e43bcaa2492c287d7af P 19e79fbaeb2f4d8f9e4cb95bc73df894 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58c281726a7d46c2b6c754c8a4fe5bf4 in term 0.
I20260924 13:47:17.261381 16101 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bb1e9376203c4e43bcaa2492c287d7af" candidate_uuid: "58c281726a7d46c2b6c754c8a4fe5bf4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b0580d290b47438a9c7cace92e848865" is_pre_election: true
I20260924 13:47:17.261451 15748 leader_election.cc:302] T bb1e9376203c4e43bcaa2492c287d7af P 58c281726a7d46c2b6c754c8a4fe5bf4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19e79fbaeb2f4d8f9e4cb95bc73df894, 58c281726a7d46c2b6c754c8a4fe5bf4; no voters:
I20260924 13:47:17.261505 16101 raft_consensus.cc:2454] T bb1e9376203c4e43bcaa2492c287d7af P b0580d290b47438a9c7cace92e848865 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58c281726a7d46c2b6c754c8a4fe5bf4 in term 0.
I20260924 13:47:17.261584 16178 raft_consensus.cc:2781] T bb1e9376203c4e43bcaa2492c287d7af P 58c281726a7d46c2b6c754c8a4fe5bf4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 13:47:17.261620 16178 raft_consensus.cc:482] T bb1e9376203c4e43bcaa2492c287d7af P 58c281726a7d46c2b6c754c8a4fe5bf4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 13:47:17.261639 16178 raft_consensus.cc:3037] T bb1e9376203c4e43bcaa2492c287d7af P 58c281726a7d46c2b6c754c8a4fe5bf4 [term 0 FOLLOWER]: Advancing to term 1
I20260924 13:47:17.262359 16178 raft_consensus.cc:504] T bb1e9376203c4e43bcaa2492c287d7af P 58c281726a7d46c2b6c754c8a4fe5bf4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58c281726a7d46c2b6c754c8a4fe5bf4" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 40235 } } peers { permanent_uuid: "19e79fbaeb2f4d8f9e4cb95bc73df894" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 43723 } } peers { permanent_uuid: "b0580d290b47438a9c7cace92e848865" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 35443 } }
I20260924 13:47:17.262485 16178 leader_election.cc:288] T bb1e9376203c4e43bcaa2492c287d7af P 58c281726a7d46c2b6c754c8a4fe5bf4 [CANDIDATE]: Term 1 election: Requested vote from peers 19e79fbaeb2f4d8f9e4cb95bc73df894 (127.13.149.194:43723), b0580d290b47438a9c7cace92e848865 (127.13.149.195:35443)
I20260924 13:47:17.262619 15959 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bb1e9376203c4e43bcaa2492c287d7af" candidate_uuid: "58c281726a7d46c2b6c754c8a4fe5bf4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19e79fbaeb2f4d8f9e4cb95bc73df894"
I20260924 13:47:17.262655 16101 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bb1e9376203c4e43bcaa2492c287d7af" candidate_uuid: "58c281726a7d46c2b6c754c8a4fe5bf4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b0580d290b47438a9c7cace92e848865"
I20260924 13:47:17.262696 15959 raft_consensus.cc:3037] T bb1e9376203c4e43bcaa2492c287d7af P 19e79fbaeb2f4d8f9e4cb95bc73df894 [term 0 FOLLOWER]: Advancing to term 1
I20260924 13:47:17.262727 16101 raft_consensus.cc:3037] T bb1e9376203c4e43bcaa2492c287d7af P b0580d290b47438a9c7cace92e848865 [term 0 FOLLOWER]: Advancing to term 1
I20260924 13:47:17.263527 16101 raft_consensus.cc:2454] T bb1e9376203c4e43bcaa2492c287d7af P b0580d290b47438a9c7cace92e848865 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58c281726a7d46c2b6c754c8a4fe5bf4 in term 1.
I20260924 13:47:17.263593 15959 raft_consensus.cc:2454] T bb1e9376203c4e43bcaa2492c287d7af P 19e79fbaeb2f4d8f9e4cb95bc73df894 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58c281726a7d46c2b6c754c8a4fe5bf4 in term 1.
I20260924 13:47:17.263721 15749 leader_election.cc:302] T bb1e9376203c4e43bcaa2492c287d7af P 58c281726a7d46c2b6c754c8a4fe5bf4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58c281726a7d46c2b6c754c8a4fe5bf4, b0580d290b47438a9c7cace92e848865; no voters:
I20260924 13:47:17.263847 16178 raft_consensus.cc:2781] T bb1e9376203c4e43bcaa2492c287d7af P 58c281726a7d46c2b6c754c8a4fe5bf4 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 13:47:17.263994 16178 raft_consensus.cc:686] T bb1e9376203c4e43bcaa2492c287d7af P 58c281726a7d46c2b6c754c8a4fe5bf4 [term 1 LEADER]: Becoming Leader. State: Replica: 58c281726a7d46c2b6c754c8a4fe5bf4, State: Running, Role: LEADER
I20260924 13:47:17.264066 16178 consensus_queue.cc:237] T bb1e9376203c4e43bcaa2492c287d7af P 58c281726a7d46c2b6c754c8a4fe5bf4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58c281726a7d46c2b6c754c8a4fe5bf4" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 40235 } } peers { permanent_uuid: "19e79fbaeb2f4d8f9e4cb95bc73df894" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 43723 } } peers { permanent_uuid: "b0580d290b47438a9c7cace92e848865" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 35443 } }
I20260924 13:47:17.264840 15657 catalog_manager.cc:5495] T bb1e9376203c4e43bcaa2492c287d7af P 58c281726a7d46c2b6c754c8a4fe5bf4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 58c281726a7d46c2b6c754c8a4fe5bf4 (127.13.149.193). New cstate: current_term: 1 leader_uuid: "58c281726a7d46c2b6c754c8a4fe5bf4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58c281726a7d46c2b6c754c8a4fe5bf4" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 40235 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "19e79fbaeb2f4d8f9e4cb95bc73df894" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 43723 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b0580d290b47438a9c7cace92e848865" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 35443 } health_report { overall_health: UNKNOWN } } }
I20260924 13:47:17.284108 16168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:17.281174 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46832 (local address 127.13.149.193:40235)
0924 13:47:17.281225 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0924 13:47:17.281227 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:47:17.281253 (+ 26us) server_negotiation.cc:407] Connection header received
0924 13:47:17.281289 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:47:17.281290 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:47:17.281311 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:47:17.281370 (+ 59us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 13:47:17.281464 (+ 94us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:17.282268 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:17.283203 (+ 935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:17.283530 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:17.283594 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:47:17.283785 (+ 191us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 13:47:17.283922 (+ 137us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 13:47:17.283984 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0924 13:47:17.284000 (+ 16us) server_negotiation.cc:299] Negotiation successful
0924 13:47:17.284022 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260924 13:47:17.289964 16223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:17.286775 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.194:43723 (local address 127.13.149.193:34435)
0924 13:47:17.286919 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0924 13:47:17.286929 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 13:47:17.286984 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:47:17.287101 (+ 117us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:47:17.287102 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:47:17.287103 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0924 13:47:17.287163 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:47:17.287164 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:47:17.288039 (+ 875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:47:17.288040 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:47:17.288708 (+ 668us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:47:17.288711 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:47:17.289263 (+ 552us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:47:17.289264 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:47:17.289376 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:47:17.289434 (+ 58us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0924 13:47:17.289656 (+ 222us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0924 13:47:17.289658 (+ 2us) client_negotiation.cc:769] Sending connection context
0924 13:47:17.289738 (+ 80us) client_negotiation.cc:240] Negotiation successful
0924 13:47:17.289831 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":74,"threads_started":1}
I20260924 13:47:17.289995 16167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:17.286853 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.149.193:34435 (local address 127.13.149.194:43723)
0924 13:47:17.286897 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0924 13:47:17.286899 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:47:17.286983 (+ 84us) server_negotiation.cc:407] Connection header received
0924 13:47:17.287029 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:47:17.287030 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:47:17.287039 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:47:17.287098 (+ 59us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 13:47:17.287210 (+ 112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:17.287931 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:17.288806 (+ 875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:17.289148 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:17.289212 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:47:17.289557 (+ 345us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 13:47:17.289596 (+ 39us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 13:47:17.289654 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0924 13:47:17.289744 (+ 90us) server_negotiation.cc:299] Negotiation successful
0924 13:47:17.289773 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260924 13:47:17.292447 16101 raft_consensus.cc:1261] T bb1e9376203c4e43bcaa2492c287d7af P b0580d290b47438a9c7cace92e848865 [term 1 FOLLOWER]: Refusing update from remote peer 58c281726a7d46c2b6c754c8a4fe5bf4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 13:47:17.292564 15959 raft_consensus.cc:1261] T bb1e9376203c4e43bcaa2492c287d7af P 19e79fbaeb2f4d8f9e4cb95bc73df894 [term 1 FOLLOWER]: Refusing update from remote peer 58c281726a7d46c2b6c754c8a4fe5bf4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 13:47:17.292769 16178 consensus_queue.cc:1035] T bb1e9376203c4e43bcaa2492c287d7af P 58c281726a7d46c2b6c754c8a4fe5bf4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b0580d290b47438a9c7cace92e848865" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 35443 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 13:47:17.293144 16219 consensus_queue.cc:1035] T bb1e9376203c4e43bcaa2492c287d7af P 58c281726a7d46c2b6c754c8a4fe5bf4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19e79fbaeb2f4d8f9e4cb95bc73df894" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 43723 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 13:47:17.301538 16168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:17.299088 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.149.194:42179 (local address 127.13.149.193:40235)
0924 13:47:17.299125 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0924 13:47:17.299126 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0924 13:47:17.299140 (+ 14us) server_negotiation.cc:407] Connection header received
0924 13:47:17.299220 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:47:17.299220 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:47:17.299232 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:47:17.299286 (+ 54us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 13:47:17.299534 (+ 248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:17.300005 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:17.300791 (+ 786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:17.301065 (+ 274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:17.301130 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:47:17.301290 (+ 160us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 13:47:17.301326 (+ 36us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 13:47:17.301379 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0924 13:47:17.301454 (+ 75us) server_negotiation.cc:299] Negotiation successful
0924 13:47:17.301476 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260924 13:47:17.301584 16012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:17.299038 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.193:40235 (local address 127.13.149.194:42179)
0924 13:47:17.299093 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0924 13:47:17.299104 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0924 13:47:17.299154 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:47:17.299290 (+ 136us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:47:17.299291 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:47:17.299293 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0924 13:47:17.299344 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:47:17.299346 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:47:17.300092 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:47:17.300093 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:47:17.300690 (+ 597us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:47:17.300693 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:47:17.301138 (+ 445us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:47:17.301140 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:47:17.301196 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:47:17.301203 (+ 7us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0924 13:47:17.301390 (+ 187us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0924 13:47:17.301393 (+ 3us) client_negotiation.cc:769] Sending connection context
0924 13:47:17.301448 (+ 55us) client_negotiation.cc:240] Negotiation successful
0924 13:47:17.301501 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32}
W20260924 13:47:17.302088 16198 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260924 13:47:18.828675 15656 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42852:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 13:47:18.828795 15656 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42852:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 13:47:18.829841 15656 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4dbf8e8088ce4e99ba854b50554eeb6a: Sending DeleteTablet for 3 replicas of tablet bb1e9376203c4e43bcaa2492c287d7af
I20260924 13:47:18.830138 15938 tablet_service.cc:1514] Processing DeleteTablet for tablet bb1e9376203c4e43bcaa2492c287d7af with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 13:47:18 UTC) from {username='kudu', principal='kudu/127.13.149.254@KRBTEST.COM'} at 127.0.0.1:53148
I20260924 13:47:18.830204 16081 tablet_service.cc:1514] Processing DeleteTablet for tablet bb1e9376203c4e43bcaa2492c287d7af with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 13:47:18 UTC) from {username='kudu', principal='kudu/127.13.149.254@KRBTEST.COM'} at 127.0.0.1:46412
I20260924 13:47:18.830458 15797 tablet_service.cc:1514] Processing DeleteTablet for tablet bb1e9376203c4e43bcaa2492c287d7af with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 13:47:18 UTC) from {username='kudu', principal='kudu/127.13.149.254@KRBTEST.COM'} at 127.0.0.1:46830
I20260924 13:47:18.830480 16251 tablet_replica.cc:330] T bb1e9376203c4e43bcaa2492c287d7af P b0580d290b47438a9c7cace92e848865: stopping tablet replica
I20260924 13:47:18.830534 16251 raft_consensus.cc:2229] T bb1e9376203c4e43bcaa2492c287d7af P b0580d290b47438a9c7cace92e848865 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 13:47:18.830564 16251 raft_consensus.cc:2258] T bb1e9376203c4e43bcaa2492c287d7af P b0580d290b47438a9c7cace92e848865 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 13:47:18.830665 16252 tablet_replica.cc:330] T bb1e9376203c4e43bcaa2492c287d7af P 58c281726a7d46c2b6c754c8a4fe5bf4: stopping tablet replica
I20260924 13:47:18.830757 16252 raft_consensus.cc:2229] T bb1e9376203c4e43bcaa2492c287d7af P 58c281726a7d46c2b6c754c8a4fe5bf4 [term 1 LEADER]: Raft consensus shutting down.
I20260924 13:47:18.830873 16252 raft_consensus.cc:2258] T bb1e9376203c4e43bcaa2492c287d7af P 58c281726a7d46c2b6c754c8a4fe5bf4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 13:47:18.830864 16251 ts_tablet_manager.cc:1905] T bb1e9376203c4e43bcaa2492c287d7af P b0580d290b47438a9c7cace92e848865: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 13:47:18.831085 16252 ts_tablet_manager.cc:1905] T bb1e9376203c4e43bcaa2492c287d7af P 58c281726a7d46c2b6c754c8a4fe5bf4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 13:47:18.833036 16250 tablet_replica.cc:330] T bb1e9376203c4e43bcaa2492c287d7af P 19e79fbaeb2f4d8f9e4cb95bc73df894: stopping tablet replica
I20260924 13:47:18.833134 16252 ts_tablet_manager.cc:1918] T bb1e9376203c4e43bcaa2492c287d7af P 58c281726a7d46c2b6c754c8a4fe5bf4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260924 13:47:18.833134 16251 ts_tablet_manager.cc:1918] T bb1e9376203c4e43bcaa2492c287d7af P b0580d290b47438a9c7cace92e848865: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260924 13:47:18.833199 16251 log.cc:1196] T bb1e9376203c4e43bcaa2492c287d7af P b0580d290b47438a9c7cace92e848865: Deleting WAL directory at /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790257629041645-13911-0/minicluster-data/ts-2/wal/wals/bb1e9376203c4e43bcaa2492c287d7af
I20260924 13:47:18.833294 16250 raft_consensus.cc:2229] T bb1e9376203c4e43bcaa2492c287d7af P 19e79fbaeb2f4d8f9e4cb95bc73df894 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 13:47:18.833328 16252 log.cc:1196] T bb1e9376203c4e43bcaa2492c287d7af P 58c281726a7d46c2b6c754c8a4fe5bf4: Deleting WAL directory at /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790257629041645-13911-0/minicluster-data/ts-0/wal/wals/bb1e9376203c4e43bcaa2492c287d7af
I20260924 13:47:18.833403 16251 ts_tablet_manager.cc:1939] T bb1e9376203c4e43bcaa2492c287d7af P b0580d290b47438a9c7cace92e848865: Deleting consensus metadata
I20260924 13:47:18.833467 16250 raft_consensus.cc:2258] T bb1e9376203c4e43bcaa2492c287d7af P 19e79fbaeb2f4d8f9e4cb95bc73df894 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 13:47:18.833551 16252 ts_tablet_manager.cc:1939] T bb1e9376203c4e43bcaa2492c287d7af P 58c281726a7d46c2b6c754c8a4fe5bf4: Deleting consensus metadata
I20260924 13:47:18.833741 16250 ts_tablet_manager.cc:1905] T bb1e9376203c4e43bcaa2492c287d7af P 19e79fbaeb2f4d8f9e4cb95bc73df894: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 13:47:18.833927 15644 catalog_manager.cc:4841] TS 58c281726a7d46c2b6c754c8a4fe5bf4 (127.13.149.193:40235): tablet bb1e9376203c4e43bcaa2492c287d7af (table test-table [id=b58fc0aa6a5c4d19be6fdd6bc6b0ff44]) successfully deleted
I20260924 13:47:18.834028 15644 catalog_manager.cc:4841] TS b0580d290b47438a9c7cace92e848865 (127.13.149.195:35443): tablet bb1e9376203c4e43bcaa2492c287d7af (table test-table [id=b58fc0aa6a5c4d19be6fdd6bc6b0ff44]) successfully deleted
I20260924 13:47:18.835517 16250 ts_tablet_manager.cc:1918] T bb1e9376203c4e43bcaa2492c287d7af P 19e79fbaeb2f4d8f9e4cb95bc73df894: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260924 13:47:18.835589 16250 log.cc:1196] T bb1e9376203c4e43bcaa2492c287d7af P 19e79fbaeb2f4d8f9e4cb95bc73df894: Deleting WAL directory at /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790257629041645-13911-0/minicluster-data/ts-1/wal/wals/bb1e9376203c4e43bcaa2492c287d7af
I20260924 13:47:18.835889 16250 ts_tablet_manager.cc:1939] T bb1e9376203c4e43bcaa2492c287d7af P 19e79fbaeb2f4d8f9e4cb95bc73df894: Deleting consensus metadata
I20260924 13:47:18.837133 15643 catalog_manager.cc:4841] TS 19e79fbaeb2f4d8f9e4cb95bc73df894 (127.13.149.194:43723): tablet bb1e9376203c4e43bcaa2492c287d7af (table test-table [id=b58fc0aa6a5c4d19be6fdd6bc6b0ff44]) successfully deleted
Sep 24 13:47:18 dist-test-slave-sjl2 krb5kdc[15612](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257638, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 24 13:47:18 dist-test-slave-sjl2 krb5kdc[15612](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257638, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.13.149.254@KRBTEST.COM
I20260924 13:47:18.855336 16260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:18.847636 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44092 (local address 127.13.149.254:40663)
0924 13:47:18.847831 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0924 13:47:18.847837 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0924 13:47:18.847847 (+ 10us) server_negotiation.cc:407] Connection header received
0924 13:47:18.847891 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:47:18.847891 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:47:18.847902 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:47:18.847970 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:47:18.848532 (+ 562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:18.849144 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:18.849978 (+ 834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:18.850187 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:18.850262 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:47:18.852592 (+ 2330us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:47:18.852593 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:47:18.852593 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 13:47:18.852601 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:47:18.854145 (+ 1544us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:47:18.854586 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:47:18.854587 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:47:18.854587 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:47:18.854652 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:47:18.854887 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:47:18.854888 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:47:18.854888 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:47:18.855008 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:47:18.855085 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0924 13:47:18.855166 (+ 81us) server_negotiation.cc:299] Negotiation successful
0924 13:47:18.855257 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":164,"thread_start_us":126,"threads_started":1}
I20260924 13:47:18.856980 15656 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44092:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 13:47:18.857152 15656 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 13:47:18.858439 16081 tablet_service.cc:1467] Processing CreateTablet for tablet 7bfd76b45a81485288d99658a1fa74db (DEFAULT_TABLE table=test-table [id=dde10896a35e4423980485ccc785542b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 13:47:18.858449 15797 tablet_service.cc:1467] Processing CreateTablet for tablet 7bfd76b45a81485288d99658a1fa74db (DEFAULT_TABLE table=test-table [id=dde10896a35e4423980485ccc785542b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 13:47:18.858578 16081 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7bfd76b45a81485288d99658a1fa74db. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 13:47:18.858579 15797 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7bfd76b45a81485288d99658a1fa74db. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 13:47:18.859956 16262 tablet_bootstrap.cc:492] T 7bfd76b45a81485288d99658a1fa74db P b0580d290b47438a9c7cace92e848865: Bootstrap starting.
I20260924 13:47:18.859997 15938 tablet_service.cc:1467] Processing CreateTablet for tablet 7bfd76b45a81485288d99658a1fa74db (DEFAULT_TABLE table=test-table [id=dde10896a35e4423980485ccc785542b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 13:47:18.860133 15938 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7bfd76b45a81485288d99658a1fa74db. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 13:47:18.860352 16262 tablet_bootstrap.cc:654] T 7bfd76b45a81485288d99658a1fa74db P b0580d290b47438a9c7cace92e848865: Neither blocks nor log segments found. Creating new log.
I20260924 13:47:18.860520 16261 tablet_bootstrap.cc:492] T 7bfd76b45a81485288d99658a1fa74db P 58c281726a7d46c2b6c754c8a4fe5bf4: Bootstrap starting.
I20260924 13:47:18.860880 16261 tablet_bootstrap.cc:654] T 7bfd76b45a81485288d99658a1fa74db P 58c281726a7d46c2b6c754c8a4fe5bf4: Neither blocks nor log segments found. Creating new log.
I20260924 13:47:18.861018 16262 tablet_bootstrap.cc:492] T 7bfd76b45a81485288d99658a1fa74db P b0580d290b47438a9c7cace92e848865: No bootstrap required, opened a new log
I20260924 13:47:18.861065 16262 ts_tablet_manager.cc:1397] T 7bfd76b45a81485288d99658a1fa74db P b0580d290b47438a9c7cace92e848865: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260924 13:47:18.861215 16262 raft_consensus.cc:348] T 7bfd76b45a81485288d99658a1fa74db P b0580d290b47438a9c7cace92e848865 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58c281726a7d46c2b6c754c8a4fe5bf4" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 40235 } } peers { permanent_uuid: "19e79fbaeb2f4d8f9e4cb95bc73df894" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 43723 } } peers { permanent_uuid: "b0580d290b47438a9c7cace92e848865" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 35443 } }
I20260924 13:47:18.861286 16262 raft_consensus.cc:374] T 7bfd76b45a81485288d99658a1fa74db P b0580d290b47438a9c7cace92e848865 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 13:47:18.861305 16262 raft_consensus.cc:729] T 7bfd76b45a81485288d99658a1fa74db P b0580d290b47438a9c7cace92e848865 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b0580d290b47438a9c7cace92e848865, State: Initialized, Role: FOLLOWER
I20260924 13:47:18.861364 16262 consensus_queue.cc:260] T 7bfd76b45a81485288d99658a1fa74db P b0580d290b47438a9c7cace92e848865 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58c281726a7d46c2b6c754c8a4fe5bf4" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 40235 } } peers { permanent_uuid: "19e79fbaeb2f4d8f9e4cb95bc73df894" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 43723 } } peers { permanent_uuid: "b0580d290b47438a9c7cace92e848865" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 35443 } }
I20260924 13:47:18.861619 16261 tablet_bootstrap.cc:492] T 7bfd76b45a81485288d99658a1fa74db P 58c281726a7d46c2b6c754c8a4fe5bf4: No bootstrap required, opened a new log
I20260924 13:47:18.861675 16261 ts_tablet_manager.cc:1397] T 7bfd76b45a81485288d99658a1fa74db P 58c281726a7d46c2b6c754c8a4fe5bf4: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260924 13:47:18.861691 16262 ts_tablet_manager.cc:1428] T 7bfd76b45a81485288d99658a1fa74db P b0580d290b47438a9c7cace92e848865: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260924 13:47:18.861853 16266 tablet_bootstrap.cc:492] T 7bfd76b45a81485288d99658a1fa74db P 19e79fbaeb2f4d8f9e4cb95bc73df894: Bootstrap starting.
I20260924 13:47:18.861791 16261 raft_consensus.cc:348] T 7bfd76b45a81485288d99658a1fa74db P 58c281726a7d46c2b6c754c8a4fe5bf4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58c281726a7d46c2b6c754c8a4fe5bf4" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 40235 } } peers { permanent_uuid: "19e79fbaeb2f4d8f9e4cb95bc73df894" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 43723 } } peers { permanent_uuid: "b0580d290b47438a9c7cace92e848865" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 35443 } }
I20260924 13:47:18.861914 16261 raft_consensus.cc:374] T 7bfd76b45a81485288d99658a1fa74db P 58c281726a7d46c2b6c754c8a4fe5bf4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 13:47:18.861940 16261 raft_consensus.cc:729] T 7bfd76b45a81485288d99658a1fa74db P 58c281726a7d46c2b6c754c8a4fe5bf4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 58c281726a7d46c2b6c754c8a4fe5bf4, State: Initialized, Role: FOLLOWER
I20260924 13:47:18.861984 16261 consensus_queue.cc:260] T 7bfd76b45a81485288d99658a1fa74db P 58c281726a7d46c2b6c754c8a4fe5bf4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58c281726a7d46c2b6c754c8a4fe5bf4" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 40235 } } peers { permanent_uuid: "19e79fbaeb2f4d8f9e4cb95bc73df894" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 43723 } } peers { permanent_uuid: "b0580d290b47438a9c7cace92e848865" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 35443 } }
I20260924 13:47:18.862110 16261 ts_tablet_manager.cc:1428] T 7bfd76b45a81485288d99658a1fa74db P 58c281726a7d46c2b6c754c8a4fe5bf4: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260924 13:47:18.862304 16266 tablet_bootstrap.cc:654] T 7bfd76b45a81485288d99658a1fa74db P 19e79fbaeb2f4d8f9e4cb95bc73df894: Neither blocks nor log segments found. Creating new log.
I20260924 13:47:18.862948 16266 tablet_bootstrap.cc:492] T 7bfd76b45a81485288d99658a1fa74db P 19e79fbaeb2f4d8f9e4cb95bc73df894: No bootstrap required, opened a new log
I20260924 13:47:18.862991 16266 ts_tablet_manager.cc:1397] T 7bfd76b45a81485288d99658a1fa74db P 19e79fbaeb2f4d8f9e4cb95bc73df894: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260924 13:47:18.863089 16266 raft_consensus.cc:348] T 7bfd76b45a81485288d99658a1fa74db P 19e79fbaeb2f4d8f9e4cb95bc73df894 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58c281726a7d46c2b6c754c8a4fe5bf4" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 40235 } } peers { permanent_uuid: "19e79fbaeb2f4d8f9e4cb95bc73df894" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 43723 } } peers { permanent_uuid: "b0580d290b47438a9c7cace92e848865" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 35443 } }
I20260924 13:47:18.863168 16266 raft_consensus.cc:374] T 7bfd76b45a81485288d99658a1fa74db P 19e79fbaeb2f4d8f9e4cb95bc73df894 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 13:47:18.863189 16266 raft_consensus.cc:729] T 7bfd76b45a81485288d99658a1fa74db P 19e79fbaeb2f4d8f9e4cb95bc73df894 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 19e79fbaeb2f4d8f9e4cb95bc73df894, State: Initialized, Role: FOLLOWER
I20260924 13:47:18.863241 16266 consensus_queue.cc:260] T 7bfd76b45a81485288d99658a1fa74db P 19e79fbaeb2f4d8f9e4cb95bc73df894 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58c281726a7d46c2b6c754c8a4fe5bf4" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 40235 } } peers { permanent_uuid: "19e79fbaeb2f4d8f9e4cb95bc73df894" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 43723 } } peers { permanent_uuid: "b0580d290b47438a9c7cace92e848865" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 35443 } }
I20260924 13:47:18.863332 16266 ts_tablet_manager.cc:1428] T 7bfd76b45a81485288d99658a1fa74db P 19e79fbaeb2f4d8f9e4cb95bc73df894: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260924 13:47:19.151120 16241 raft_consensus.cc:482] T 7bfd76b45a81485288d99658a1fa74db P 58c281726a7d46c2b6c754c8a4fe5bf4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 13:47:19.151226 16241 raft_consensus.cc:504] T 7bfd76b45a81485288d99658a1fa74db P 58c281726a7d46c2b6c754c8a4fe5bf4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58c281726a7d46c2b6c754c8a4fe5bf4" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 40235 } } peers { permanent_uuid: "19e79fbaeb2f4d8f9e4cb95bc73df894" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 43723 } } peers { permanent_uuid: "b0580d290b47438a9c7cace92e848865" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 35443 } }
I20260924 13:47:19.151396 16241 leader_election.cc:288] T 7bfd76b45a81485288d99658a1fa74db P 58c281726a7d46c2b6c754c8a4fe5bf4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 19e79fbaeb2f4d8f9e4cb95bc73df894 (127.13.149.194:43723), b0580d290b47438a9c7cace92e848865 (127.13.149.195:35443)
I20260924 13:47:19.151620 15959 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7bfd76b45a81485288d99658a1fa74db" candidate_uuid: "58c281726a7d46c2b6c754c8a4fe5bf4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19e79fbaeb2f4d8f9e4cb95bc73df894" is_pre_election: true
I20260924 13:47:19.151643 16100 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7bfd76b45a81485288d99658a1fa74db" candidate_uuid: "58c281726a7d46c2b6c754c8a4fe5bf4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b0580d290b47438a9c7cace92e848865" is_pre_election: true
I20260924 13:47:19.151712 15959 raft_consensus.cc:2454] T 7bfd76b45a81485288d99658a1fa74db P 19e79fbaeb2f4d8f9e4cb95bc73df894 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58c281726a7d46c2b6c754c8a4fe5bf4 in term 0.
I20260924 13:47:19.151739 16100 raft_consensus.cc:2454] T 7bfd76b45a81485288d99658a1fa74db P b0580d290b47438a9c7cace92e848865 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58c281726a7d46c2b6c754c8a4fe5bf4 in term 0.
I20260924 13:47:19.151891 15748 leader_election.cc:302] T 7bfd76b45a81485288d99658a1fa74db P 58c281726a7d46c2b6c754c8a4fe5bf4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19e79fbaeb2f4d8f9e4cb95bc73df894, 58c281726a7d46c2b6c754c8a4fe5bf4; no voters:
I20260924 13:47:19.151981 16241 raft_consensus.cc:2781] T 7bfd76b45a81485288d99658a1fa74db P 58c281726a7d46c2b6c754c8a4fe5bf4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 13:47:19.152022 16241 raft_consensus.cc:482] T 7bfd76b45a81485288d99658a1fa74db P 58c281726a7d46c2b6c754c8a4fe5bf4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 13:47:19.152040 16241 raft_consensus.cc:3037] T 7bfd76b45a81485288d99658a1fa74db P 58c281726a7d46c2b6c754c8a4fe5bf4 [term 0 FOLLOWER]: Advancing to term 1
I20260924 13:47:19.152727 16241 raft_consensus.cc:504] T 7bfd76b45a81485288d99658a1fa74db P 58c281726a7d46c2b6c754c8a4fe5bf4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58c281726a7d46c2b6c754c8a4fe5bf4" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 40235 } } peers { permanent_uuid: "19e79fbaeb2f4d8f9e4cb95bc73df894" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 43723 } } peers { permanent_uuid: "b0580d290b47438a9c7cace92e848865" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 35443 } }
I20260924 13:47:19.152868 16241 leader_election.cc:288] T 7bfd76b45a81485288d99658a1fa74db P 58c281726a7d46c2b6c754c8a4fe5bf4 [CANDIDATE]: Term 1 election: Requested vote from peers 19e79fbaeb2f4d8f9e4cb95bc73df894 (127.13.149.194:43723), b0580d290b47438a9c7cace92e848865 (127.13.149.195:35443)
I20260924 13:47:19.153065 15959 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7bfd76b45a81485288d99658a1fa74db" candidate_uuid: "58c281726a7d46c2b6c754c8a4fe5bf4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19e79fbaeb2f4d8f9e4cb95bc73df894"
I20260924 13:47:19.153092 16100 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7bfd76b45a81485288d99658a1fa74db" candidate_uuid: "58c281726a7d46c2b6c754c8a4fe5bf4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b0580d290b47438a9c7cace92e848865"
I20260924 13:47:19.153174 16100 raft_consensus.cc:3037] T 7bfd76b45a81485288d99658a1fa74db P b0580d290b47438a9c7cace92e848865 [term 0 FOLLOWER]: Advancing to term 1
I20260924 13:47:19.153174 15959 raft_consensus.cc:3037] T 7bfd76b45a81485288d99658a1fa74db P 19e79fbaeb2f4d8f9e4cb95bc73df894 [term 0 FOLLOWER]: Advancing to term 1
I20260924 13:47:19.153961 15959 raft_consensus.cc:2454] T 7bfd76b45a81485288d99658a1fa74db P 19e79fbaeb2f4d8f9e4cb95bc73df894 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58c281726a7d46c2b6c754c8a4fe5bf4 in term 1.
I20260924 13:47:19.153962 16100 raft_consensus.cc:2454] T 7bfd76b45a81485288d99658a1fa74db P b0580d290b47438a9c7cace92e848865 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58c281726a7d46c2b6c754c8a4fe5bf4 in term 1.
I20260924 13:47:19.154184 15748 leader_election.cc:302] T 7bfd76b45a81485288d99658a1fa74db P 58c281726a7d46c2b6c754c8a4fe5bf4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19e79fbaeb2f4d8f9e4cb95bc73df894, 58c281726a7d46c2b6c754c8a4fe5bf4; no voters:
I20260924 13:47:19.154309 16241 raft_consensus.cc:2781] T 7bfd76b45a81485288d99658a1fa74db P 58c281726a7d46c2b6c754c8a4fe5bf4 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 13:47:19.154382 16241 raft_consensus.cc:686] T 7bfd76b45a81485288d99658a1fa74db P 58c281726a7d46c2b6c754c8a4fe5bf4 [term 1 LEADER]: Becoming Leader. State: Replica: 58c281726a7d46c2b6c754c8a4fe5bf4, State: Running, Role: LEADER
I20260924 13:47:19.154439 16241 consensus_queue.cc:237] T 7bfd76b45a81485288d99658a1fa74db P 58c281726a7d46c2b6c754c8a4fe5bf4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58c281726a7d46c2b6c754c8a4fe5bf4" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 40235 } } peers { permanent_uuid: "19e79fbaeb2f4d8f9e4cb95bc73df894" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 43723 } } peers { permanent_uuid: "b0580d290b47438a9c7cace92e848865" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 35443 } }
I20260924 13:47:19.155076 15656 catalog_manager.cc:5495] T 7bfd76b45a81485288d99658a1fa74db P 58c281726a7d46c2b6c754c8a4fe5bf4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 58c281726a7d46c2b6c754c8a4fe5bf4 (127.13.149.193). New cstate: current_term: 1 leader_uuid: "58c281726a7d46c2b6c754c8a4fe5bf4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58c281726a7d46c2b6c754c8a4fe5bf4" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 40235 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "19e79fbaeb2f4d8f9e4cb95bc73df894" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 43723 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b0580d290b47438a9c7cace92e848865" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 35443 } health_report { overall_health: UNKNOWN } } }
I20260924 13:47:19.219549 16270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:19.217126 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55628 (local address 127.13.149.193:40235)
0924 13:47:19.217273 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0924 13:47:19.217275 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:47:19.217280 (+ 5us) server_negotiation.cc:407] Connection header received
0924 13:47:19.217316 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:47:19.217316 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:47:19.217326 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:47:19.217378 (+ 52us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 13:47:19.217503 (+ 125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:19.218092 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:19.218883 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:19.219118 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:19.219177 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:47:19.219308 (+ 131us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 13:47:19.219345 (+ 37us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 13:47:19.219389 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0924 13:47:19.219440 (+ 51us) server_negotiation.cc:299] Negotiation successful
0924 13:47:19.219461 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123,"thread_start_us":85,"threads_started":1}
I20260924 13:47:19.221837 15959 raft_consensus.cc:1261] T 7bfd76b45a81485288d99658a1fa74db P 19e79fbaeb2f4d8f9e4cb95bc73df894 [term 1 FOLLOWER]: Refusing update from remote peer 58c281726a7d46c2b6c754c8a4fe5bf4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 13:47:19.221837 16100 raft_consensus.cc:1261] T 7bfd76b45a81485288d99658a1fa74db P b0580d290b47438a9c7cace92e848865 [term 1 FOLLOWER]: Refusing update from remote peer 58c281726a7d46c2b6c754c8a4fe5bf4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 13:47:19.222090 16244 consensus_queue.cc:1035] T 7bfd76b45a81485288d99658a1fa74db P 58c281726a7d46c2b6c754c8a4fe5bf4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b0580d290b47438a9c7cace92e848865" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 35443 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 13:47:19.222229 16241 consensus_queue.cc:1035] T 7bfd76b45a81485288d99658a1fa74db P 58c281726a7d46c2b6c754c8a4fe5bf4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19e79fbaeb2f4d8f9e4cb95bc73df894" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 43723 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 13:47:19.231611 15657 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44092:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 13:47:19.231732 15657 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44092:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 13:47:19.232405 15657 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4dbf8e8088ce4e99ba854b50554eeb6a: Sending DeleteTablet for 3 replicas of tablet 7bfd76b45a81485288d99658a1fa74db
I20260924 13:47:19.232638 15797 tablet_service.cc:1514] Processing DeleteTablet for tablet 7bfd76b45a81485288d99658a1fa74db with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 13:47:19 UTC) from {username='kudu', principal='kudu/127.13.149.254@KRBTEST.COM'} at 127.0.0.1:46830
I20260924 13:47:19.232671 16081 tablet_service.cc:1514] Processing DeleteTablet for tablet 7bfd76b45a81485288d99658a1fa74db with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 13:47:19 UTC) from {username='kudu', principal='kudu/127.13.149.254@KRBTEST.COM'} at 127.0.0.1:46412
I20260924 13:47:19.232751 16252 tablet_replica.cc:330] T 7bfd76b45a81485288d99658a1fa74db P 58c281726a7d46c2b6c754c8a4fe5bf4: stopping tablet replica
I20260924 13:47:19.232740 15938 tablet_service.cc:1514] Processing DeleteTablet for tablet 7bfd76b45a81485288d99658a1fa74db with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 13:47:19 UTC) from {username='kudu', principal='kudu/127.13.149.254@KRBTEST.COM'} at 127.0.0.1:53148
I20260924 13:47:19.232777 16251 tablet_replica.cc:330] T 7bfd76b45a81485288d99658a1fa74db P b0580d290b47438a9c7cace92e848865: stopping tablet replica
I20260924 13:47:19.232800 16252 raft_consensus.cc:2229] T 7bfd76b45a81485288d99658a1fa74db P 58c281726a7d46c2b6c754c8a4fe5bf4 [term 1 LEADER]: Raft consensus shutting down.
I20260924 13:47:19.232822 16250 tablet_replica.cc:330] T 7bfd76b45a81485288d99658a1fa74db P 19e79fbaeb2f4d8f9e4cb95bc73df894: stopping tablet replica
I20260924 13:47:19.232823 16251 raft_consensus.cc:2229] T 7bfd76b45a81485288d99658a1fa74db P b0580d290b47438a9c7cace92e848865 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 13:47:19.232865 16251 raft_consensus.cc:2258] T 7bfd76b45a81485288d99658a1fa74db P b0580d290b47438a9c7cace92e848865 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 13:47:19.232870 16250 raft_consensus.cc:2229] T 7bfd76b45a81485288d99658a1fa74db P 19e79fbaeb2f4d8f9e4cb95bc73df894 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 13:47:19.232892 16252 raft_consensus.cc:2258] T 7bfd76b45a81485288d99658a1fa74db P 58c281726a7d46c2b6c754c8a4fe5bf4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 13:47:19.232909 16250 raft_consensus.cc:2258] T 7bfd76b45a81485288d99658a1fa74db P 19e79fbaeb2f4d8f9e4cb95bc73df894 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 13:47:19.233188 16252 ts_tablet_manager.cc:1905] T 7bfd76b45a81485288d99658a1fa74db P 58c281726a7d46c2b6c754c8a4fe5bf4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 13:47:19.233331 16251 ts_tablet_manager.cc:1905] T 7bfd76b45a81485288d99658a1fa74db P b0580d290b47438a9c7cace92e848865: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 13:47:19.233301 15657 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44092:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 13:47:19.233453 15657 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 13:47:19.234903 16252 ts_tablet_manager.cc:1918] T 7bfd76b45a81485288d99658a1fa74db P 58c281726a7d46c2b6c754c8a4fe5bf4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260924 13:47:19.234903 16251 ts_tablet_manager.cc:1918] T 7bfd76b45a81485288d99658a1fa74db P b0580d290b47438a9c7cace92e848865: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260924 13:47:19.234966 16252 log.cc:1196] T 7bfd76b45a81485288d99658a1fa74db P 58c281726a7d46c2b6c754c8a4fe5bf4: Deleting WAL directory at /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790257629041645-13911-0/minicluster-data/ts-0/wal/wals/7bfd76b45a81485288d99658a1fa74db
I20260924 13:47:19.234966 16251 log.cc:1196] T 7bfd76b45a81485288d99658a1fa74db P b0580d290b47438a9c7cace92e848865: Deleting WAL directory at /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790257629041645-13911-0/minicluster-data/ts-2/wal/wals/7bfd76b45a81485288d99658a1fa74db
I20260924 13:47:19.234943 16250 ts_tablet_manager.cc:1905] T 7bfd76b45a81485288d99658a1fa74db P 19e79fbaeb2f4d8f9e4cb95bc73df894: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 13:47:19.235168 16252 ts_tablet_manager.cc:1939] T 7bfd76b45a81485288d99658a1fa74db P 58c281726a7d46c2b6c754c8a4fe5bf4: Deleting consensus metadata
I20260924 13:47:19.235168 16251 ts_tablet_manager.cc:1939] T 7bfd76b45a81485288d99658a1fa74db P b0580d290b47438a9c7cace92e848865: Deleting consensus metadata
I20260924 13:47:19.235491 15644 catalog_manager.cc:4841] TS b0580d290b47438a9c7cace92e848865 (127.13.149.195:35443): tablet 7bfd76b45a81485288d99658a1fa74db (table test-table [id=dde10896a35e4423980485ccc785542b]) successfully deleted
I20260924 13:47:19.235565 15644 catalog_manager.cc:4841] TS 58c281726a7d46c2b6c754c8a4fe5bf4 (127.13.149.193:40235): tablet 7bfd76b45a81485288d99658a1fa74db (table test-table [id=dde10896a35e4423980485ccc785542b]) successfully deleted
I20260924 13:47:19.236205 16250 ts_tablet_manager.cc:1918] T 7bfd76b45a81485288d99658a1fa74db P 19e79fbaeb2f4d8f9e4cb95bc73df894: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260924 13:47:19.236261 16250 log.cc:1196] T 7bfd76b45a81485288d99658a1fa74db P 19e79fbaeb2f4d8f9e4cb95bc73df894: Deleting WAL directory at /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790257629041645-13911-0/minicluster-data/ts-1/wal/wals/7bfd76b45a81485288d99658a1fa74db
I20260924 13:47:19.236431 16250 ts_tablet_manager.cc:1939] T 7bfd76b45a81485288d99658a1fa74db P 19e79fbaeb2f4d8f9e4cb95bc73df894: Deleting consensus metadata
I20260924 13:47:19.236979 15643 catalog_manager.cc:4841] TS 19e79fbaeb2f4d8f9e4cb95bc73df894 (127.13.149.194:43723): tablet 7bfd76b45a81485288d99658a1fa74db (table test-table [id=dde10896a35e4423980485ccc785542b]) successfully deleted
I20260924 13:47:19.237536 15797 tablet_service.cc:1467] Processing CreateTablet for tablet 43a0463fbc4848c1b67290025b5ac958 (DEFAULT_TABLE table=test-table [id=d80bc9b209e248c0a44ff5b429da79f7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 13:47:19.237548 16081 tablet_service.cc:1467] Processing CreateTablet for tablet 43a0463fbc4848c1b67290025b5ac958 (DEFAULT_TABLE table=test-table [id=d80bc9b209e248c0a44ff5b429da79f7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 13:47:19.237653 16081 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 43a0463fbc4848c1b67290025b5ac958. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 13:47:19.237658 15797 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 43a0463fbc4848c1b67290025b5ac958. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 13:47:19.238552 16262 tablet_bootstrap.cc:492] T 43a0463fbc4848c1b67290025b5ac958 P b0580d290b47438a9c7cace92e848865: Bootstrap starting.
I20260924 13:47:19.238583 15938 tablet_service.cc:1467] Processing CreateTablet for tablet 43a0463fbc4848c1b67290025b5ac958 (DEFAULT_TABLE table=test-table [id=d80bc9b209e248c0a44ff5b429da79f7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 13:47:19.238680 15938 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 43a0463fbc4848c1b67290025b5ac958. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 13:47:19.239007 16262 tablet_bootstrap.cc:654] T 43a0463fbc4848c1b67290025b5ac958 P b0580d290b47438a9c7cace92e848865: Neither blocks nor log segments found. Creating new log.
I20260924 13:47:19.239411 16261 tablet_bootstrap.cc:492] T 43a0463fbc4848c1b67290025b5ac958 P 58c281726a7d46c2b6c754c8a4fe5bf4: Bootstrap starting.
I20260924 13:47:19.239599 16262 tablet_bootstrap.cc:492] T 43a0463fbc4848c1b67290025b5ac958 P b0580d290b47438a9c7cace92e848865: No bootstrap required, opened a new log
I20260924 13:47:19.239648 16262 ts_tablet_manager.cc:1397] T 43a0463fbc4848c1b67290025b5ac958 P b0580d290b47438a9c7cace92e848865: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260924 13:47:19.239765 16261 tablet_bootstrap.cc:654] T 43a0463fbc4848c1b67290025b5ac958 P 58c281726a7d46c2b6c754c8a4fe5bf4: Neither blocks nor log segments found. Creating new log.
I20260924 13:47:19.239841 16262 raft_consensus.cc:348] T 43a0463fbc4848c1b67290025b5ac958 P b0580d290b47438a9c7cace92e848865 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58c281726a7d46c2b6c754c8a4fe5bf4" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 40235 } } peers { permanent_uuid: "19e79fbaeb2f4d8f9e4cb95bc73df894" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 43723 } } peers { permanent_uuid: "b0580d290b47438a9c7cace92e848865" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 35443 } }
I20260924 13:47:19.239923 16262 raft_consensus.cc:374] T 43a0463fbc4848c1b67290025b5ac958 P b0580d290b47438a9c7cace92e848865 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 13:47:19.239941 16262 raft_consensus.cc:729] T 43a0463fbc4848c1b67290025b5ac958 P b0580d290b47438a9c7cace92e848865 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b0580d290b47438a9c7cace92e848865, State: Initialized, Role: FOLLOWER
I20260924 13:47:19.239966 16266 tablet_bootstrap.cc:492] T 43a0463fbc4848c1b67290025b5ac958 P 19e79fbaeb2f4d8f9e4cb95bc73df894: Bootstrap starting.
I20260924 13:47:19.239980 16262 consensus_queue.cc:260] T 43a0463fbc4848c1b67290025b5ac958 P b0580d290b47438a9c7cace92e848865 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58c281726a7d46c2b6c754c8a4fe5bf4" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 40235 } } peers { permanent_uuid: "19e79fbaeb2f4d8f9e4cb95bc73df894" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 43723 } } peers { permanent_uuid: "b0580d290b47438a9c7cace92e848865" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 35443 } }
I20260924 13:47:19.240061 16262 ts_tablet_manager.cc:1428] T 43a0463fbc4848c1b67290025b5ac958 P b0580d290b47438a9c7cace92e848865: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260924 13:47:19.240259 16266 tablet_bootstrap.cc:654] T 43a0463fbc4848c1b67290025b5ac958 P 19e79fbaeb2f4d8f9e4cb95bc73df894: Neither blocks nor log segments found. Creating new log.
I20260924 13:47:19.240290 16261 tablet_bootstrap.cc:492] T 43a0463fbc4848c1b67290025b5ac958 P 58c281726a7d46c2b6c754c8a4fe5bf4: No bootstrap required, opened a new log
I20260924 13:47:19.240334 16261 ts_tablet_manager.cc:1397] T 43a0463fbc4848c1b67290025b5ac958 P 58c281726a7d46c2b6c754c8a4fe5bf4: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260924 13:47:19.240437 16261 raft_consensus.cc:348] T 43a0463fbc4848c1b67290025b5ac958 P 58c281726a7d46c2b6c754c8a4fe5bf4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58c281726a7d46c2b6c754c8a4fe5bf4" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 40235 } } peers { permanent_uuid: "19e79fbaeb2f4d8f9e4cb95bc73df894" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 43723 } } peers { permanent_uuid: "b0580d290b47438a9c7cace92e848865" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 35443 } }
I20260924 13:47:19.240497 16261 raft_consensus.cc:374] T 43a0463fbc4848c1b67290025b5ac958 P 58c281726a7d46c2b6c754c8a4fe5bf4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 13:47:19.240515 16261 raft_consensus.cc:729] T 43a0463fbc4848c1b67290025b5ac958 P 58c281726a7d46c2b6c754c8a4fe5bf4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 58c281726a7d46c2b6c754c8a4fe5bf4, State: Initialized, Role: FOLLOWER
I20260924 13:47:19.240556 16261 consensus_queue.cc:260] T 43a0463fbc4848c1b67290025b5ac958 P 58c281726a7d46c2b6c754c8a4fe5bf4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58c281726a7d46c2b6c754c8a4fe5bf4" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 40235 } } peers { permanent_uuid: "19e79fbaeb2f4d8f9e4cb95bc73df894" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 43723 } } peers { permanent_uuid: "b0580d290b47438a9c7cace92e848865" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 35443 } }
I20260924 13:47:19.240643 16261 ts_tablet_manager.cc:1428] T 43a0463fbc4848c1b67290025b5ac958 P 58c281726a7d46c2b6c754c8a4fe5bf4: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260924 13:47:19.240939 16266 tablet_bootstrap.cc:492] T 43a0463fbc4848c1b67290025b5ac958 P 19e79fbaeb2f4d8f9e4cb95bc73df894: No bootstrap required, opened a new log
I20260924 13:47:19.240986 16266 ts_tablet_manager.cc:1397] T 43a0463fbc4848c1b67290025b5ac958 P 19e79fbaeb2f4d8f9e4cb95bc73df894: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260924 13:47:19.241088 16266 raft_consensus.cc:348] T 43a0463fbc4848c1b67290025b5ac958 P 19e79fbaeb2f4d8f9e4cb95bc73df894 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58c281726a7d46c2b6c754c8a4fe5bf4" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 40235 } } peers { permanent_uuid: "19e79fbaeb2f4d8f9e4cb95bc73df894" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 43723 } } peers { permanent_uuid: "b0580d290b47438a9c7cace92e848865" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 35443 } }
I20260924 13:47:19.241146 16266 raft_consensus.cc:374] T 43a0463fbc4848c1b67290025b5ac958 P 19e79fbaeb2f4d8f9e4cb95bc73df894 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 13:47:19.241164 16266 raft_consensus.cc:729] T 43a0463fbc4848c1b67290025b5ac958 P 19e79fbaeb2f4d8f9e4cb95bc73df894 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 19e79fbaeb2f4d8f9e4cb95bc73df894, State: Initialized, Role: FOLLOWER
I20260924 13:47:19.241205 16266 consensus_queue.cc:260] T 43a0463fbc4848c1b67290025b5ac958 P 19e79fbaeb2f4d8f9e4cb95bc73df894 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58c281726a7d46c2b6c754c8a4fe5bf4" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 40235 } } peers { permanent_uuid: "19e79fbaeb2f4d8f9e4cb95bc73df894" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 43723 } } peers { permanent_uuid: "b0580d290b47438a9c7cace92e848865" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 35443 } }
I20260924 13:47:19.241290 16266 ts_tablet_manager.cc:1428] T 43a0463fbc4848c1b67290025b5ac958 P 19e79fbaeb2f4d8f9e4cb95bc73df894: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260924 13:47:19.463883 16265 raft_consensus.cc:482] T 43a0463fbc4848c1b67290025b5ac958 P b0580d290b47438a9c7cace92e848865 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 13:47:19.463987 16265 raft_consensus.cc:504] T 43a0463fbc4848c1b67290025b5ac958 P b0580d290b47438a9c7cace92e848865 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58c281726a7d46c2b6c754c8a4fe5bf4" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 40235 } } peers { permanent_uuid: "19e79fbaeb2f4d8f9e4cb95bc73df894" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 43723 } } peers { permanent_uuid: "b0580d290b47438a9c7cace92e848865" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 35443 } }
I20260924 13:47:19.464345 16265 leader_election.cc:288] T 43a0463fbc4848c1b67290025b5ac958 P b0580d290b47438a9c7cace92e848865 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 58c281726a7d46c2b6c754c8a4fe5bf4 (127.13.149.193:40235), 19e79fbaeb2f4d8f9e4cb95bc73df894 (127.13.149.194:43723)
I20260924 13:47:19.467635 16270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:19.464660 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.149.195:35895 (local address 127.13.149.193:40235)
0924 13:47:19.464709 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0924 13:47:19.464712 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 13:47:19.464809 (+ 97us) server_negotiation.cc:407] Connection header received
0924 13:47:19.464866 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:47:19.464867 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:47:19.464879 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:47:19.464962 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 13:47:19.465096 (+ 134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:19.465574 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:19.466866 (+ 1292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:19.467298 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:19.467366 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:47:19.467377 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0924 13:47:19.467520 (+ 143us) server_negotiation.cc:299] Negotiation successful
0924 13:47:19.467561 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260924 13:47:19.467650 16279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:19.464618 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.149.195:37201 (local address 127.13.149.194:43723)
0924 13:47:19.464712 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0924 13:47:19.464714 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:47:19.464796 (+ 82us) server_negotiation.cc:407] Connection header received
0924 13:47:19.464827 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:47:19.464828 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:47:19.464839 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:47:19.464902 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 13:47:19.465074 (+ 172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:19.465564 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:19.466891 (+ 1327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:19.467302 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:19.467368 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:47:19.467377 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0924 13:47:19.467518 (+ 141us) server_negotiation.cc:299] Negotiation successful
0924 13:47:19.467560 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76,"thread_start_us":43,"threads_started":1}
I20260924 13:47:19.467684 16277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:19.464551 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.193:40235 (local address 127.13.149.195:35895)
0924 13:47:19.464704 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0924 13:47:19.464749 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0924 13:47:19.464810 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:47:19.464963 (+ 153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:47:19.464963 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:47:19.464966 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 13:47:19.465042 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:47:19.465044 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:47:19.465655 (+ 611us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:47:19.465656 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:47:19.466781 (+ 1125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:47:19.466785 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:47:19.467380 (+ 595us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:47:19.467381 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:47:19.467457 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:47:19.467462 (+ 5us) client_negotiation.cc:769] Sending connection context
0924 13:47:19.467508 (+ 46us) client_negotiation.cc:240] Negotiation successful
0924 13:47:19.467558 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"mutex_wait_us":48,"spinlock_wait_cycles":896,"thread_start_us":95,"threads_started":1}
I20260924 13:47:19.467686 16278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:19.464550 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.194:43723 (local address 127.13.149.195:37201)
0924 13:47:19.464697 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0924 13:47:19.464709 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0924 13:47:19.464774 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:47:19.464935 (+ 161us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:47:19.464936 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:47:19.464938 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 13:47:19.465011 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:47:19.465014 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:47:19.465673 (+ 659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:47:19.465674 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:47:19.466807 (+ 1133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:47:19.466810 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:47:19.467377 (+ 567us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:47:19.467379 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:47:19.467458 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:47:19.467461 (+ 3us) client_negotiation.cc:769] Sending connection context
0924 13:47:19.467507 (+ 46us) client_negotiation.cc:240] Negotiation successful
0924 13:47:19.467558 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"mutex_wait_us":100,"thread_start_us":65,"threads_started":1}
I20260924 13:47:19.467999 15959 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43a0463fbc4848c1b67290025b5ac958" candidate_uuid: "b0580d290b47438a9c7cace92e848865" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19e79fbaeb2f4d8f9e4cb95bc73df894" is_pre_election: true
I20260924 13:47:19.467999 15817 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43a0463fbc4848c1b67290025b5ac958" candidate_uuid: "b0580d290b47438a9c7cace92e848865" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58c281726a7d46c2b6c754c8a4fe5bf4" is_pre_election: true
I20260924 13:47:19.468101 15817 raft_consensus.cc:2454] T 43a0463fbc4848c1b67290025b5ac958 P 58c281726a7d46c2b6c754c8a4fe5bf4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b0580d290b47438a9c7cace92e848865 in term 0.
I20260924 13:47:19.468101 15959 raft_consensus.cc:2454] T 43a0463fbc4848c1b67290025b5ac958 P 19e79fbaeb2f4d8f9e4cb95bc73df894 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b0580d290b47438a9c7cace92e848865 in term 0.
I20260924 13:47:19.468259 16032 leader_election.cc:302] T 43a0463fbc4848c1b67290025b5ac958 P b0580d290b47438a9c7cace92e848865 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19e79fbaeb2f4d8f9e4cb95bc73df894, b0580d290b47438a9c7cace92e848865; no voters:
I20260924 13:47:19.468431 16265 raft_consensus.cc:2781] T 43a0463fbc4848c1b67290025b5ac958 P b0580d290b47438a9c7cace92e848865 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 13:47:19.468462 16265 raft_consensus.cc:482] T 43a0463fbc4848c1b67290025b5ac958 P b0580d290b47438a9c7cace92e848865 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 13:47:19.468475 16265 raft_consensus.cc:3037] T 43a0463fbc4848c1b67290025b5ac958 P b0580d290b47438a9c7cace92e848865 [term 0 FOLLOWER]: Advancing to term 1
I20260924 13:47:19.469170 16265 raft_consensus.cc:504] T 43a0463fbc4848c1b67290025b5ac958 P b0580d290b47438a9c7cace92e848865 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58c281726a7d46c2b6c754c8a4fe5bf4" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 40235 } } peers { permanent_uuid: "19e79fbaeb2f4d8f9e4cb95bc73df894" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 43723 } } peers { permanent_uuid: "b0580d290b47438a9c7cace92e848865" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 35443 } }
I20260924 13:47:19.469302 16265 leader_election.cc:288] T 43a0463fbc4848c1b67290025b5ac958 P b0580d290b47438a9c7cace92e848865 [CANDIDATE]: Term 1 election: Requested vote from peers 58c281726a7d46c2b6c754c8a4fe5bf4 (127.13.149.193:40235), 19e79fbaeb2f4d8f9e4cb95bc73df894 (127.13.149.194:43723)
I20260924 13:47:19.469472 15817 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43a0463fbc4848c1b67290025b5ac958" candidate_uuid: "b0580d290b47438a9c7cace92e848865" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58c281726a7d46c2b6c754c8a4fe5bf4"
I20260924 13:47:19.469491 15959 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43a0463fbc4848c1b67290025b5ac958" candidate_uuid: "b0580d290b47438a9c7cace92e848865" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19e79fbaeb2f4d8f9e4cb95bc73df894"
I20260924 13:47:19.469549 15817 raft_consensus.cc:3037] T 43a0463fbc4848c1b67290025b5ac958 P 58c281726a7d46c2b6c754c8a4fe5bf4 [term 0 FOLLOWER]: Advancing to term 1
I20260924 13:47:19.469576 15959 raft_consensus.cc:3037] T 43a0463fbc4848c1b67290025b5ac958 P 19e79fbaeb2f4d8f9e4cb95bc73df894 [term 0 FOLLOWER]: Advancing to term 1
I20260924 13:47:19.470108 15817 raft_consensus.cc:2454] T 43a0463fbc4848c1b67290025b5ac958 P 58c281726a7d46c2b6c754c8a4fe5bf4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b0580d290b47438a9c7cace92e848865 in term 1.
I20260924 13:47:19.470237 15959 raft_consensus.cc:2454] T 43a0463fbc4848c1b67290025b5ac958 P 19e79fbaeb2f4d8f9e4cb95bc73df894 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b0580d290b47438a9c7cace92e848865 in term 1.
I20260924 13:47:19.470294 16033 leader_election.cc:302] T 43a0463fbc4848c1b67290025b5ac958 P b0580d290b47438a9c7cace92e848865 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58c281726a7d46c2b6c754c8a4fe5bf4, b0580d290b47438a9c7cace92e848865; no voters:
I20260924 13:47:19.470382 16265 raft_consensus.cc:2781] T 43a0463fbc4848c1b67290025b5ac958 P b0580d290b47438a9c7cace92e848865 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 13:47:19.470520 16265 raft_consensus.cc:686] T 43a0463fbc4848c1b67290025b5ac958 P b0580d290b47438a9c7cace92e848865 [term 1 LEADER]: Becoming Leader. State: Replica: b0580d290b47438a9c7cace92e848865, State: Running, Role: LEADER
I20260924 13:47:19.470584 16265 consensus_queue.cc:237] T 43a0463fbc4848c1b67290025b5ac958 P b0580d290b47438a9c7cace92e848865 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58c281726a7d46c2b6c754c8a4fe5bf4" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 40235 } } peers { permanent_uuid: "19e79fbaeb2f4d8f9e4cb95bc73df894" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 43723 } } peers { permanent_uuid: "b0580d290b47438a9c7cace92e848865" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 35443 } }
I20260924 13:47:19.471305 15657 catalog_manager.cc:5495] T 43a0463fbc4848c1b67290025b5ac958 P b0580d290b47438a9c7cace92e848865 reported cstate change: term changed from 0 to 1, leader changed from <none> to b0580d290b47438a9c7cace92e848865 (127.13.149.195). New cstate: current_term: 1 leader_uuid: "b0580d290b47438a9c7cace92e848865" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58c281726a7d46c2b6c754c8a4fe5bf4" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 40235 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "19e79fbaeb2f4d8f9e4cb95bc73df894" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 43723 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b0580d290b47438a9c7cace92e848865" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 35443 } health_report { overall_health: HEALTHY } } }
I20260924 13:47:19.528344 16282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:19.525767 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52964 (local address 127.13.149.195:35443)
0924 13:47:19.525952 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0924 13:47:19.525953 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0924 13:47:19.525959 (+ 6us) server_negotiation.cc:407] Connection header received
0924 13:47:19.525995 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:47:19.525996 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:47:19.526014 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:47:19.526086 (+ 72us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 13:47:19.526208 (+ 122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:19.526715 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:19.527453 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:19.527699 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:19.527759 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:47:19.527939 (+ 180us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 13:47:19.528094 (+ 155us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 13:47:19.528157 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0924 13:47:19.528195 (+ 38us) server_negotiation.cc:299] Negotiation successful
0924 13:47:19.528241 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":157,"thread_start_us":116,"threads_started":1}
I20260924 13:47:19.533603 16279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:19.530966 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.149.195:51735 (local address 127.13.149.194:43723)
0924 13:47:19.531020 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0924 13:47:19.531022 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:47:19.531048 (+ 26us) server_negotiation.cc:407] Connection header received
0924 13:47:19.531070 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:47:19.531070 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:47:19.531079 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:47:19.531114 (+ 35us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 13:47:19.531236 (+ 122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:19.531699 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:19.533005 (+ 1306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:19.533200 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:19.533257 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:47:19.533426 (+ 169us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 13:47:19.533451 (+ 25us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 13:47:19.533482 (+ 31us) server_negotiation.cc:1035] Waiting for connection context
0924 13:47:19.533523 (+ 41us) server_negotiation.cc:299] Negotiation successful
0924 13:47:19.533542 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260924 13:47:19.533627 16285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:19.530811 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.194:43723 (local address 127.13.149.195:51735)
0924 13:47:19.530961 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0924 13:47:19.530973 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0924 13:47:19.531018 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:47:19.531116 (+ 98us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:47:19.531117 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:47:19.531118 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0924 13:47:19.531174 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:47:19.531175 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:47:19.531798 (+ 623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:47:19.531798 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:47:19.532885 (+ 1087us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:47:19.532888 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:47:19.533294 (+ 406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:47:19.533295 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:47:19.533347 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:47:19.533362 (+ 15us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0924 13:47:19.533491 (+ 129us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0924 13:47:19.533493 (+ 2us) client_negotiation.cc:769] Sending connection context
0924 13:47:19.533521 (+ 28us) client_negotiation.cc:240] Negotiation successful
0924 13:47:19.533557 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":69,"threads_started":1}
I20260924 13:47:19.535648 15817 raft_consensus.cc:1261] T 43a0463fbc4848c1b67290025b5ac958 P 58c281726a7d46c2b6c754c8a4fe5bf4 [term 1 FOLLOWER]: Refusing update from remote peer b0580d290b47438a9c7cace92e848865: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 13:47:19.535648 15959 raft_consensus.cc:1261] T 43a0463fbc4848c1b67290025b5ac958 P 19e79fbaeb2f4d8f9e4cb95bc73df894 [term 1 FOLLOWER]: Refusing update from remote peer b0580d290b47438a9c7cace92e848865: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 13:47:19.535969 16280 consensus_queue.cc:1035] T 43a0463fbc4848c1b67290025b5ac958 P b0580d290b47438a9c7cace92e848865 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58c281726a7d46c2b6c754c8a4fe5bf4" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 40235 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 13:47:19.536094 16265 consensus_queue.cc:1035] T 43a0463fbc4848c1b67290025b5ac958 P b0580d290b47438a9c7cace92e848865 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19e79fbaeb2f4d8f9e4cb95bc73df894" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 43723 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 13:47:19.543860 16282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:19.541683 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.149.194:52397 (local address 127.13.149.195:35443)
0924 13:47:19.541728 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0924 13:47:19.541730 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:47:19.541739 (+ 9us) server_negotiation.cc:407] Connection header received
0924 13:47:19.541804 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:47:19.541805 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:47:19.541814 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:47:19.541857 (+ 43us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 13:47:19.541948 (+ 91us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:19.542437 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:19.543169 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:19.543470 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:19.543534 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:47:19.543647 (+ 113us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 13:47:19.543675 (+ 28us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 13:47:19.543745 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0924 13:47:19.543783 (+ 38us) server_negotiation.cc:299] Negotiation successful
0924 13:47:19.543802 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260924 13:47:19.543880 16290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:19.541558 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.195:35443 (local address 127.13.149.194:52397)
0924 13:47:19.541677 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0924 13:47:19.541689 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0924 13:47:19.541747 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:47:19.541860 (+ 113us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:47:19.541860 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:47:19.541862 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0924 13:47:19.541908 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:47:19.541910 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:47:19.542489 (+ 579us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:47:19.542489 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:47:19.543105 (+ 616us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:47:19.543107 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:47:19.543539 (+ 432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:47:19.543540 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:47:19.543587 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:47:19.543592 (+ 5us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0924 13:47:19.543750 (+ 158us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0924 13:47:19.543751 (+ 1us) client_negotiation.cc:769] Sending connection context
0924 13:47:19.543780 (+ 29us) client_negotiation.cc:240] Negotiation successful
0924 13:47:19.543818 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":93,"thread_start_us":76,"threads_started":1}
W20260924 13:47:19.544185 16233 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260924 13:47:21.063541 15657 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44092:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 13:47:21.063647 15657 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44092:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 13:47:21.064510 15657 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4dbf8e8088ce4e99ba854b50554eeb6a: Sending DeleteTablet for 3 replicas of tablet 43a0463fbc4848c1b67290025b5ac958
I20260924 13:47:21.064770 16081 tablet_service.cc:1514] Processing DeleteTablet for tablet 43a0463fbc4848c1b67290025b5ac958 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 13:47:21 UTC) from {username='kudu', principal='kudu/127.13.149.254@KRBTEST.COM'} at 127.0.0.1:46412
I20260924 13:47:21.064823 15797 tablet_service.cc:1514] Processing DeleteTablet for tablet 43a0463fbc4848c1b67290025b5ac958 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 13:47:21 UTC) from {username='kudu', principal='kudu/127.13.149.254@KRBTEST.COM'} at 127.0.0.1:46830
I20260924 13:47:21.064952 13911 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJN9WRF/build/release/bin/kudu with pid 15731
I20260924 13:47:21.065563 15938 tablet_service.cc:1514] Processing DeleteTablet for tablet 43a0463fbc4848c1b67290025b5ac958 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 13:47:21 UTC) from {username='kudu', principal='kudu/127.13.149.254@KRBTEST.COM'} at 127.0.0.1:53148
I20260924 13:47:21.065770 16313 tablet_replica.cc:330] T 43a0463fbc4848c1b67290025b5ac958 P b0580d290b47438a9c7cace92e848865: stopping tablet replica
I20260924 13:47:21.065825 16313 raft_consensus.cc:2229] T 43a0463fbc4848c1b67290025b5ac958 P b0580d290b47438a9c7cace92e848865 [term 1 LEADER]: Raft consensus shutting down.
I20260924 13:47:21.065918 16313 raft_consensus.cc:2258] T 43a0463fbc4848c1b67290025b5ac958 P b0580d290b47438a9c7cace92e848865 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 13:47:21.066010 16315 tablet_replica.cc:330] T 43a0463fbc4848c1b67290025b5ac958 P 19e79fbaeb2f4d8f9e4cb95bc73df894: stopping tablet replica
I20260924 13:47:21.066066 16315 raft_consensus.cc:2229] T 43a0463fbc4848c1b67290025b5ac958 P 19e79fbaeb2f4d8f9e4cb95bc73df894 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 13:47:21.066107 16315 raft_consensus.cc:2258] T 43a0463fbc4848c1b67290025b5ac958 P 19e79fbaeb2f4d8f9e4cb95bc73df894 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 13:47:21.066419 16315 ts_tablet_manager.cc:1905] T 43a0463fbc4848c1b67290025b5ac958 P 19e79fbaeb2f4d8f9e4cb95bc73df894: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 13:47:21.068166 16313 ts_tablet_manager.cc:1905] T 43a0463fbc4848c1b67290025b5ac958 P b0580d290b47438a9c7cace92e848865: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 13:47:21.068372 16315 ts_tablet_manager.cc:1918] T 43a0463fbc4848c1b67290025b5ac958 P 19e79fbaeb2f4d8f9e4cb95bc73df894: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260924 13:47:21.068429 16315 log.cc:1196] T 43a0463fbc4848c1b67290025b5ac958 P 19e79fbaeb2f4d8f9e4cb95bc73df894: Deleting WAL directory at /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790257629041645-13911-0/minicluster-data/ts-1/wal/wals/43a0463fbc4848c1b67290025b5ac958
I20260924 13:47:21.068641 16315 ts_tablet_manager.cc:1939] T 43a0463fbc4848c1b67290025b5ac958 P 19e79fbaeb2f4d8f9e4cb95bc73df894: Deleting consensus metadata
I20260924 13:47:21.069342 15643 catalog_manager.cc:4841] TS 19e79fbaeb2f4d8f9e4cb95bc73df894 (127.13.149.194:43723): tablet 43a0463fbc4848c1b67290025b5ac958 (table test-table [id=d80bc9b209e248c0a44ff5b429da79f7]) successfully deleted
I20260924 13:47:21.069697 16313 ts_tablet_manager.cc:1918] T 43a0463fbc4848c1b67290025b5ac958 P b0580d290b47438a9c7cace92e848865: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260924 13:47:21.069757 16313 log.cc:1196] T 43a0463fbc4848c1b67290025b5ac958 P b0580d290b47438a9c7cace92e848865: Deleting WAL directory at /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790257629041645-13911-0/minicluster-data/ts-2/wal/wals/43a0463fbc4848c1b67290025b5ac958
I20260924 13:47:21.069975 16313 ts_tablet_manager.cc:1939] T 43a0463fbc4848c1b67290025b5ac958 P b0580d290b47438a9c7cace92e848865: Deleting consensus metadata
I20260924 13:47:21.070344 15644 catalog_manager.cc:4841] TS b0580d290b47438a9c7cace92e848865 (127.13.149.195:35443): tablet 43a0463fbc4848c1b67290025b5ac958 (table test-table [id=d80bc9b209e248c0a44ff5b429da79f7]) successfully deleted
W20260924 13:47:21.071861 15644 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.13.149.193:40235 (error 108)
I20260924 13:47:21.071980 13911 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJN9WRF/build/release/bin/kudu with pid 15872
I20260924 13:47:21.073352 16317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:21.072860 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.193:40235 (local address 127.0.0.1:55634)
0924 13:47:21.073257 (+ 397us) negotiation.cc:107] Waiting for socket to connect
0924 13:47:21.073306 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.149.193:40235: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":371,"thread_start_us":96,"threads_started":1}
W20260924 13:47:21.073510 15644 catalog_manager.cc:4584] TS 58c281726a7d46c2b6c754c8a4fe5bf4 (127.13.149.193:40235): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 43a0463fbc4848c1b67290025b5ac958: Network error: Client connection negotiation failed: client connection to 127.13.149.193:40235: connect: Connection refused (error 111)
I20260924 13:47:21.077582 13911 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJN9WRF/build/release/bin/kudu with pid 16015
I20260924 13:47:21.082630 13911 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJN9WRF/build/release/bin/kudu with pid 15625
2026-09-24T13:47:21Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (5959 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790257629041645-13911-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 13:47:21 dist-test-slave-sjl2 krb5kdc[16320](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 13:47:21 dist-test-slave-sjl2 krb5kdc[16320](info): set up 2 sockets
Sep 24 13:47:21 dist-test-slave-sjl2 krb5kdc[16320](info): commencing operation
krb5kdc: starting...
W20260924 13:47:22.041705 13911 mini_kdc.cc:121] Time spent starting KDC: real 0.938s user 0.002s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790257629041645-13911-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790257629041645-13911-0/krb5kdc/test-user.keytab.
Sep 24 13:47:22 dist-test-slave-sjl2 krb5kdc[16320](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257642, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T13:47:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T13:47:22Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.149.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.149.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.149.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.254.keytab.
Entry for principal kudu/127.13.149.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.254.keytab.
I20260924 13:47:22.159890 13911 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJN9WRF/build/release/bin/kudu
/tmp/dist-test-taskJN9WRF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790257629041645-13911-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790257629041645-13911-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790257629041645-13911-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790257629041645-13911-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.149.254:41937
--webserver_interface=127.13.149.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.254.keytab
--principal=kudu/127.13.149.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.149.212:43199
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.149.254:41937
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790257629041645-13911-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790257629041645-13911-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790257629041645-13911-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 13:47:22.244977 16334 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 13:47:22.245224 16334 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 13:47:22.245257 16334 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 13:47:22.246554 16334 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 13:47:22.246606 16334 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 13:47:22.246625 16334 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 13:47:22.246640 16334 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 13:47:22.246655 16334 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 13:47:22.248279 16334 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.149.212:43199
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790257629041645-13911-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790257629041645-13911-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.149.254:41937
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.254.keytab
--principal=kudu/127.13.149.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.149.254:41937
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790257629041645-13911-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.149.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16334
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790257629041645-13911-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 12:29:37 UTC on ccycloud.quasar-iqzbxn.root.comops.site
build id 4670106
I20260924 13:47:22.248550 16334 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 13:47:22.248772 16334 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 13:47:22.254056 16339 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 13:47:22.254045 16340 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 13:47:22.254065 16342 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 13:47:22.254076 16334 server_base.cc:1034] running on GCE node
I20260924 13:47:22.254530 16334 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 13:47:22.254849 16334 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 13:47:22.256021 16334 hybrid_clock.cc:648] HybridClock initialized: now 1790257642255998 us; error 38 us; skew 500 ppm
Sep 24 13:47:22 dist-test-slave-sjl2 krb5kdc[16320](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257642, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.149.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 13:47:22.258709 16334 init.cc:377] Logged in from keytab as kudu/127.13.149.254@KRBTEST.COM (short username kudu)
I20260924 13:47:22.259236 16334 webserver.cc:460] Webserver started at http://127.13.149.254:35917/ using document root <none> and password file <none>
I20260924 13:47:22.259454 16334 fs_manager.cc:359] Metadata directory not provided
I20260924 13:47:22.259507 16334 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 13:47:22.259649 16334 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 13:47:22.260789 16334 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790257629041645-13911-0/minicluster-data/master-0/data/instance:
uuid: "16c0ae354887440793f6104de2595027"
format_stamp: "Formatted at 2026-09-24 13:47:22 on dist-test-slave-sjl2"
server_key: "f6769966a52fd39d1c762e3f367c690f"
server_key_iv: "bad9bf52b050dfca03814380fe5ced27"
server_key_version: "encryptionkey@0"
I20260924 13:47:22.261224 16334 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790257629041645-13911-0/minicluster-data/master-0/wal/instance:
uuid: "16c0ae354887440793f6104de2595027"
format_stamp: "Formatted at 2026-09-24 13:47:22 on dist-test-slave-sjl2"
server_key: "f6769966a52fd39d1c762e3f367c690f"
server_key_iv: "bad9bf52b050dfca03814380fe5ced27"
server_key_version: "encryptionkey@0"
I20260924 13:47:22.262593 16334 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260924 13:47:22.263314 16349 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 13:47:22.263550 16334 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260924 13:47:22.263648 16334 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790257629041645-13911-0/minicluster-data/master-0/data,/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790257629041645-13911-0/minicluster-data/master-0/wal
uuid: "16c0ae354887440793f6104de2595027"
format_stamp: "Formatted at 2026-09-24 13:47:22 on dist-test-slave-sjl2"
server_key: "f6769966a52fd39d1c762e3f367c690f"
server_key_iv: "bad9bf52b050dfca03814380fe5ced27"
server_key_version: "encryptionkey@0"
I20260924 13:47:22.263717 16334 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790257629041645-13911-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790257629041645-13911-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790257629041645-13911-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 13:47:22.274237 16334 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 13:47:22.276634 16334 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 13:47:22.276819 16334 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 13:47:22.280601 16334 rpc_server.cc:307] RPC server started. Bound to: 127.13.149.254:41937
I20260924 13:47:22.280634 16401 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.149.254:41937 every 8 connection(s)
I20260924 13:47:22.281016 16334 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790257629041645-13911-0/minicluster-data/master-0/data/info.pb
I20260924 13:47:22.281889 16402 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 13:47:22.284221 16402 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 16c0ae354887440793f6104de2595027: Bootstrap starting.
I20260924 13:47:22.284812 16402 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 16c0ae354887440793f6104de2595027: Neither blocks nor log segments found. Creating new log.
I20260924 13:47:22.285080 13911 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJN9WRF/build/release/bin/kudu as pid 16334
I20260924 13:47:22.285153 13911 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790257629041645-13911-0/minicluster-data/master-0/wal/instance
I20260924 13:47:22.285126 16402 log.cc:824] T 00000000000000000000000000000000 P 16c0ae354887440793f6104de2595027: Log is configured to *not* fsync() on all Append() calls
I20260924 13:47:22.285306 13911 external_mini_cluster.cc:1428] Setting key dc5cb34c8f05f9b7365c04151c564325
I20260924 13:47:22.285925 16402 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 16c0ae354887440793f6104de2595027: No bootstrap required, opened a new log
I20260924 13:47:22.288406 16402 raft_consensus.cc:348] T 00000000000000000000000000000000 P 16c0ae354887440793f6104de2595027 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "16c0ae354887440793f6104de2595027" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 41937 } }
I20260924 13:47:22.288597 16402 raft_consensus.cc:374] T 00000000000000000000000000000000 P 16c0ae354887440793f6104de2595027 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 13:47:22.288633 16402 raft_consensus.cc:729] T 00000000000000000000000000000000 P 16c0ae354887440793f6104de2595027 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 16c0ae354887440793f6104de2595027, State: Initialized, Role: FOLLOWER
I20260924 13:47:22.288765 16402 consensus_queue.cc:260] T 00000000000000000000000000000000 P 16c0ae354887440793f6104de2595027 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "16c0ae354887440793f6104de2595027" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 41937 } }
I20260924 13:47:22.288844 16402 raft_consensus.cc:388] T 00000000000000000000000000000000 P 16c0ae354887440793f6104de2595027 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 13:47:22.288879 16402 raft_consensus.cc:482] T 00000000000000000000000000000000 P 16c0ae354887440793f6104de2595027 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 13:47:22.288955 16402 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 16c0ae354887440793f6104de2595027 [term 0 FOLLOWER]: Advancing to term 1
I20260924 13:47:22.289676 16402 raft_consensus.cc:504] T 00000000000000000000000000000000 P 16c0ae354887440793f6104de2595027 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "16c0ae354887440793f6104de2595027" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 41937 } }
I20260924 13:47:22.289772 16402 leader_election.cc:302] T 00000000000000000000000000000000 P 16c0ae354887440793f6104de2595027 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 16c0ae354887440793f6104de2595027; no voters:
I20260924 13:47:22.289935 16402 leader_election.cc:288] T 00000000000000000000000000000000 P 16c0ae354887440793f6104de2595027 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 13:47:22.289975 16407 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 16c0ae354887440793f6104de2595027 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 13:47:22.290184 16407 raft_consensus.cc:686] T 00000000000000000000000000000000 P 16c0ae354887440793f6104de2595027 [term 1 LEADER]: Becoming Leader. State: Replica: 16c0ae354887440793f6104de2595027, State: Running, Role: LEADER
I20260924 13:47:22.290315 16402 sys_catalog.cc:564] T 00000000000000000000000000000000 P 16c0ae354887440793f6104de2595027 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 13:47:22.290318 16407 consensus_queue.cc:237] T 00000000000000000000000000000000 P 16c0ae354887440793f6104de2595027 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "16c0ae354887440793f6104de2595027" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 41937 } }
I20260924 13:47:22.290849 16409 sys_catalog.cc:455] T 00000000000000000000000000000000 P 16c0ae354887440793f6104de2595027 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 16c0ae354887440793f6104de2595027. Latest consensus state: current_term: 1 leader_uuid: "16c0ae354887440793f6104de2595027" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "16c0ae354887440793f6104de2595027" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 41937 } } }
I20260924 13:47:22.291414 16409 sys_catalog.cc:458] T 00000000000000000000000000000000 P 16c0ae354887440793f6104de2595027 [sys.catalog]: This master's current role is: LEADER
I20260924 13:47:22.290830 16408 sys_catalog.cc:455] T 00000000000000000000000000000000 P 16c0ae354887440793f6104de2595027 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "16c0ae354887440793f6104de2595027" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "16c0ae354887440793f6104de2595027" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 41937 } } }
I20260924 13:47:22.291672 16422 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
Sep 24 13:47:22 dist-test-slave-sjl2 krb5kdc[16320](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257642, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.149.254@KRBTEST.COM
I20260924 13:47:22.292460 16422 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 13:47:22.291782 16408 sys_catalog.cc:458] T 00000000000000000000000000000000 P 16c0ae354887440793f6104de2595027 [sys.catalog]: This master's current role is: LEADER
I20260924 13:47:22.294158 16422 catalog_manager.cc:1347] Generated new cluster ID: b5f8502593f24d5094e2c525d62fda99
I20260924 13:47:22.294205 16422 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 13:47:22.296634 16406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:22.285808 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52980 (local address 127.13.149.254:41937)
0924 13:47:22.285961 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0924 13:47:22.285971 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0924 13:47:22.286004 (+ 33us) server_negotiation.cc:407] Connection header received
0924 13:47:22.286459 (+ 455us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:47:22.286466 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:47:22.286517 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:47:22.286630 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:47:22.287407 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:22.288354 (+ 947us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:22.289382 (+ 1028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:22.289897 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:22.289997 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:47:22.292798 (+ 2801us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:47:22.292802 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:47:22.292803 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 13:47:22.292816 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:47:22.294731 (+ 1915us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:47:22.295163 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:47:22.295166 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:47:22.295167 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:47:22.295286 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:47:22.295508 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:47:22.295509 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:47:22.295509 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:47:22.295827 (+ 318us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:47:22.295956 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0924 13:47:22.296250 (+ 294us) server_negotiation.cc:299] Negotiation successful
0924 13:47:22.296401 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":105,"thread_start_us":64,"threads_started":1}
I20260924 13:47:22.319494 16422 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 13:47:22.320317 16422 catalog_manager.cc:1504] Loading token signing keys...
I20260924 13:47:22.324249 16422 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 16c0ae354887440793f6104de2595027: Generated new TSK 0
I20260924 13:47:22.324476 16422 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.13.149.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.149.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.149.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.193.keytab.
Entry for principal kudu/127.13.149.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.193.keytab.
I20260924 13:47:22.351670 13911 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJN9WRF/build/release/bin/kudu
/tmp/dist-test-taskJN9WRF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790257629041645-13911-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790257629041645-13911-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790257629041645-13911-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790257629041645-13911-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.149.193:0
--local_ip_for_outbound_sockets=127.13.149.193
--webserver_interface=127.13.149.193
--webserver_port=0
--tserver_master_addrs=127.13.149.254:41937
--keytab_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.193.keytab
--principal=kudu/127.13.149.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.149.212:43199
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790257629041645-13911-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790257629041645-13911-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790257629041645-13911-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 13:47:22.436992 16428 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 13:47:22.437227 16428 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 13:47:22.437252 16428 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 13:47:22.438565 16428 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 13:47:22.438622 16428 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 13:47:22.438652 16428 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.149.193
I20260924 13:47:22.440281 16428 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.149.212:43199
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790257629041645-13911-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790257629041645-13911-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.193.keytab
--principal=kudu/127.13.149.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.149.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790257629041645-13911-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.149.193
--webserver_port=0
--tserver_master_addrs=127.13.149.254:41937
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16428
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.149.193
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790257629041645-13911-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 12:29:37 UTC on ccycloud.quasar-iqzbxn.root.comops.site
build id 4670106
I20260924 13:47:22.440542 16428 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 13:47:22.440799 16428 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 13:47:22.446136 16436 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 13:47:22.446238 16433 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 13:47:22.446303 16428 server_base.cc:1034] running on GCE node
W20260924 13:47:22.446174 16434 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 13:47:22.446807 16428 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 13:47:22.447081 16428 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 13:47:22.448256 16428 hybrid_clock.cc:648] HybridClock initialized: now 1790257642448238 us; error 37 us; skew 500 ppm
Sep 24 13:47:22 dist-test-slave-sjl2 krb5kdc[16320](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257642, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.149.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 13:47:22.451315 16428 init.cc:377] Logged in from keytab as kudu/127.13.149.193@KRBTEST.COM (short username kudu)
I20260924 13:47:22.451761 16428 webserver.cc:460] Webserver started at http://127.13.149.193:36873/ using document root <none> and password file <none>
I20260924 13:47:22.451982 16428 fs_manager.cc:359] Metadata directory not provided
I20260924 13:47:22.452031 16428 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 13:47:22.452140 16428 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 13:47:22.453287 16428 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790257629041645-13911-0/minicluster-data/ts-0/data/instance:
uuid: "96070ac78c5f40de8f732e3b141e5656"
format_stamp: "Formatted at 2026-09-24 13:47:22 on dist-test-slave-sjl2"
server_key: "f213ad12921bd07bbefabe852d381e26"
server_key_iv: "3c4f242073993ec753b96fc9f1b8420e"
server_key_version: "encryptionkey@0"
I20260924 13:47:22.453625 16428 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790257629041645-13911-0/minicluster-data/ts-0/wal/instance:
uuid: "96070ac78c5f40de8f732e3b141e5656"
format_stamp: "Formatted at 2026-09-24 13:47:22 on dist-test-slave-sjl2"
server_key: "f213ad12921bd07bbefabe852d381e26"
server_key_iv: "3c4f242073993ec753b96fc9f1b8420e"
server_key_version: "encryptionkey@0"
I20260924 13:47:22.454969 16428 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260924 13:47:22.455873 16443 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 13:47:22.456069 16428 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260924 13:47:22.456182 16428 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790257629041645-13911-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790257629041645-13911-0/minicluster-data/ts-0/wal
uuid: "96070ac78c5f40de8f732e3b141e5656"
format_stamp: "Formatted at 2026-09-24 13:47:22 on dist-test-slave-sjl2"
server_key: "f213ad12921bd07bbefabe852d381e26"
server_key_iv: "3c4f242073993ec753b96fc9f1b8420e"
server_key_version: "encryptionkey@0"
I20260924 13:47:22.456259 16428 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790257629041645-13911-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790257629041645-13911-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790257629041645-13911-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 13:47:22.468910 16428 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 13:47:22.471776 16428 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 13:47:22.471912 16428 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 13:47:22.472164 16428 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 13:47:22.472499 16428 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 13:47:22.472533 16428 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 13:47:22.472558 16428 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 13:47:22.472570 16428 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 13:47:22.478324 16428 rpc_server.cc:307] RPC server started. Bound to: 127.13.149.193:33673
I20260924 13:47:22.478368 16557 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.149.193:33673 every 8 connection(s)
I20260924 13:47:22.478693 16428 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790257629041645-13911-0/minicluster-data/ts-0/data/info.pb
Sep 24 13:47:22 dist-test-slave-sjl2 krb5kdc[16320](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257642, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.149.193@KRBTEST.COM for kudu/127.13.149.254@KRBTEST.COM
I20260924 13:47:22.487125 13911 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJN9WRF/build/release/bin/kudu as pid 16428
I20260924 13:47:22.487233 13911 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790257629041645-13911-0/minicluster-data/ts-0/wal/instance
I20260924 13:47:22.487416 13911 external_mini_cluster.cc:1428] Setting key d8398738b831fa5194d094af0712340c
I20260924 13:47:22.487773 16406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:22.479412 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.149.193:58101 (local address 127.13.149.254:41937)
0924 13:47:22.479473 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0924 13:47:22.479475 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:47:22.479680 (+ 205us) server_negotiation.cc:407] Connection header received
0924 13:47:22.480180 (+ 500us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:47:22.480181 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:47:22.480192 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:47:22.480240 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:47:22.481271 (+ 1031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:22.481779 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:22.482766 (+ 987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:22.482975 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:22.483029 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:47:22.485145 (+ 2116us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:47:22.485147 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:47:22.485147 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 13:47:22.485154 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:47:22.486406 (+ 1252us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:47:22.486786 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:47:22.486786 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:47:22.486787 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:47:22.486839 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:47:22.487080 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:47:22.487081 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:47:22.487082 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:47:22.487241 (+ 159us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:47:22.487330 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0924 13:47:22.487599 (+ 269us) server_negotiation.cc:299] Negotiation successful
0924 13:47:22.487685 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260924 13:47:22.488034 16560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:22.479412 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:41937 (local address 127.13.149.193:58101)
0924 13:47:22.479574 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0924 13:47:22.479611 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0924 13:47:22.480093 (+ 482us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:47:22.480304 (+ 211us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:47:22.480307 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:47:22.480901 (+ 594us) client_negotiation.cc:189] Negotiated authn=SASL
0924 13:47:22.481185 (+ 284us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:47:22.481190 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:47:22.481853 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:47:22.481854 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:47:22.482682 (+ 828us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:47:22.482684 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:47:22.483039 (+ 355us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:47:22.483040 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:47:22.483143 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:47:22.483636 (+ 493us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 13:47:22.483645 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0924 13:47:22.485043 (+ 1398us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 13:47:22.486484 (+ 1441us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 13:47:22.486492 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 13:47:22.486494 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 13:47:22.486726 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 13:47:22.486887 (+ 161us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 13:47:22.486888 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 13:47:22.486888 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 13:47:22.487001 (+ 113us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 13:47:22.487321 (+ 320us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 13:47:22.487327 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 13:47:22.487468 (+ 141us) client_negotiation.cc:769] Sending connection context
0924 13:47:22.487577 (+ 109us) client_negotiation.cc:240] Negotiation successful
0924 13:47:22.487791 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":84,"threads_started":1}
I20260924 13:47:22.488624 16558 heartbeater.cc:347] Connected to a master server at 127.13.149.254:41937
I20260924 13:47:22.488754 16558 heartbeater.cc:464] Registering TS with master...
I20260924 13:47:22.488974 16558 heartbeater.cc:511] Master 127.13.149.254:41937 requested a full tablet report, sending...
I20260924 13:47:22.489585 16366 ts_manager.cc:194] Registered new tserver with Master: 96070ac78c5f40de8f732e3b141e5656 (127.13.149.193:33673)
I20260924 13:47:22.490266 16366 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.149.193@KRBTEST.COM'} at 127.13.149.193:58101
WARNING: no policy specified for kudu/127.13.149.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.149.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.149.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.194.keytab.
Entry for principal kudu/127.13.149.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.194.keytab.
I20260924 13:47:22.513352 13911 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJN9WRF/build/release/bin/kudu
/tmp/dist-test-taskJN9WRF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790257629041645-13911-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790257629041645-13911-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790257629041645-13911-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790257629041645-13911-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.149.194:0
--local_ip_for_outbound_sockets=127.13.149.194
--webserver_interface=127.13.149.194
--webserver_port=0
--tserver_master_addrs=127.13.149.254:41937
--keytab_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.194.keytab
--principal=kudu/127.13.149.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.149.212:43199
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790257629041645-13911-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790257629041645-13911-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790257629041645-13911-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 13:47:22.596330 16563 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 13:47:22.596556 16563 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 13:47:22.596581 16563 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 13:47:22.597954 16563 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 13:47:22.598006 16563 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 13:47:22.598037 16563 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.149.194
I20260924 13:47:22.599588 16563 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.149.212:43199
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790257629041645-13911-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790257629041645-13911-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.194.keytab
--principal=kudu/127.13.149.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.149.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790257629041645-13911-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.149.194
--webserver_port=0
--tserver_master_addrs=127.13.149.254:41937
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16563
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.149.194
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790257629041645-13911-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 12:29:37 UTC on ccycloud.quasar-iqzbxn.root.comops.site
build id 4670106
I20260924 13:47:22.599840 16563 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 13:47:22.600060 16563 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 13:47:22.605198 16571 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 13:47:22.605168 16568 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 13:47:22.605211 16563 server_base.cc:1034] running on GCE node
W20260924 13:47:22.605180 16569 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 13:47:22.605636 16563 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 13:47:22.605880 16563 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 13:47:22.607035 16563 hybrid_clock.cc:648] HybridClock initialized: now 1790257642607018 us; error 33 us; skew 500 ppm
Sep 24 13:47:22 dist-test-slave-sjl2 krb5kdc[16320](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257642, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.149.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 13:47:22.609874 16563 init.cc:377] Logged in from keytab as kudu/127.13.149.194@KRBTEST.COM (short username kudu)
I20260924 13:47:22.610472 16563 webserver.cc:460] Webserver started at http://127.13.149.194:41555/ using document root <none> and password file <none>
I20260924 13:47:22.610747 16563 fs_manager.cc:359] Metadata directory not provided
I20260924 13:47:22.610805 16563 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 13:47:22.610936 16563 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 13:47:22.612584 16563 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790257629041645-13911-0/minicluster-data/ts-1/data/instance:
uuid: "e895f941237a48f998c154fa936cbe80"
format_stamp: "Formatted at 2026-09-24 13:47:22 on dist-test-slave-sjl2"
server_key: "3aaf7b8acef7eaafa16bd8d9c1840690"
server_key_iv: "8ced2dcb55aca966d952f866ae6abed9"
server_key_version: "encryptionkey@0"
I20260924 13:47:22.613076 16563 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790257629041645-13911-0/minicluster-data/ts-1/wal/instance:
uuid: "e895f941237a48f998c154fa936cbe80"
format_stamp: "Formatted at 2026-09-24 13:47:22 on dist-test-slave-sjl2"
server_key: "3aaf7b8acef7eaafa16bd8d9c1840690"
server_key_iv: "8ced2dcb55aca966d952f866ae6abed9"
server_key_version: "encryptionkey@0"
I20260924 13:47:22.614665 16563 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260924 13:47:22.615468 16578 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 13:47:22.615727 16563 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260924 13:47:22.615821 16563 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790257629041645-13911-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790257629041645-13911-0/minicluster-data/ts-1/wal
uuid: "e895f941237a48f998c154fa936cbe80"
format_stamp: "Formatted at 2026-09-24 13:47:22 on dist-test-slave-sjl2"
server_key: "3aaf7b8acef7eaafa16bd8d9c1840690"
server_key_iv: "8ced2dcb55aca966d952f866ae6abed9"
server_key_version: "encryptionkey@0"
I20260924 13:47:22.615895 16563 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790257629041645-13911-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790257629041645-13911-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790257629041645-13911-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 13:47:22.624634 16563 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 13:47:22.627275 16563 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 13:47:22.627449 16563 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 13:47:22.627683 16563 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 13:47:22.628031 16563 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 13:47:22.628073 16563 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 13:47:22.628111 16563 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 13:47:22.628135 16563 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 13:47:22.634109 16563 rpc_server.cc:307] RPC server started. Bound to: 127.13.149.194:36673
I20260924 13:47:22.634151 16692 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.149.194:36673 every 8 connection(s)
I20260924 13:47:22.634481 16563 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790257629041645-13911-0/minicluster-data/ts-1/data/info.pb
I20260924 13:47:22.638294 13911 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJN9WRF/build/release/bin/kudu as pid 16563
I20260924 13:47:22.638396 13911 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790257629041645-13911-0/minicluster-data/ts-1/wal/instance
I20260924 13:47:22.638562 13911 external_mini_cluster.cc:1428] Setting key 108551a0e4ddc0858b41f2f3ebae2cba
Sep 24 13:47:22 dist-test-slave-sjl2 krb5kdc[16320](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257642, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.149.194@KRBTEST.COM for kudu/127.13.149.254@KRBTEST.COM
I20260924 13:47:22.643741 16406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:22.635287 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.149.194:34511 (local address 127.13.149.254:41937)
0924 13:47:22.635370 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0924 13:47:22.635373 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 13:47:22.635529 (+ 156us) server_negotiation.cc:407] Connection header received
0924 13:47:22.636047 (+ 518us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:47:22.636049 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:47:22.636061 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:47:22.636115 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:47:22.637120 (+ 1005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:22.637613 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:22.638641 (+ 1028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:22.638840 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:22.638898 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:47:22.641020 (+ 2122us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:47:22.641022 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:47:22.641023 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 13:47:22.641032 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:47:22.642258 (+ 1226us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:47:22.642680 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:47:22.642682 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:47:22.642683 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:47:22.642757 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:47:22.643057 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:47:22.643058 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:47:22.643058 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:47:22.643182 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:47:22.643234 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0924 13:47:22.643581 (+ 347us) server_negotiation.cc:299] Negotiation successful
0924 13:47:22.643656 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260924 13:47:22.644021 16695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:22.635193 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:41937 (local address 127.13.149.194:34511)
0924 13:47:22.635399 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0924 13:47:22.635445 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0924 13:47:22.635943 (+ 498us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:47:22.636189 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:47:22.636193 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:47:22.636779 (+ 586us) client_negotiation.cc:189] Negotiated authn=SASL
0924 13:47:22.637029 (+ 250us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:47:22.637034 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:47:22.637700 (+ 666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:47:22.637701 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:47:22.638561 (+ 860us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:47:22.638564 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:47:22.638914 (+ 350us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:47:22.638914 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:47:22.639049 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:47:22.639556 (+ 507us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 13:47:22.639565 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0924 13:47:22.640900 (+ 1335us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 13:47:22.642356 (+ 1456us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 13:47:22.642366 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 13:47:22.642369 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0924 13:47:22.642599 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 13:47:22.642850 (+ 251us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 13:47:22.642851 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 13:47:22.642851 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 13:47:22.642993 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 13:47:22.643241 (+ 248us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 13:47:22.643247 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 13:47:22.643441 (+ 194us) client_negotiation.cc:769] Sending connection context
0924 13:47:22.643566 (+ 125us) client_negotiation.cc:240] Negotiation successful
0924 13:47:22.643749 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":76,"threads_started":1}
I20260924 13:47:22.644590 16693 heartbeater.cc:347] Connected to a master server at 127.13.149.254:41937
I20260924 13:47:22.644706 16693 heartbeater.cc:464] Registering TS with master...
I20260924 13:47:22.644990 16693 heartbeater.cc:511] Master 127.13.149.254:41937 requested a full tablet report, sending...
I20260924 13:47:22.645665 16366 ts_manager.cc:194] Registered new tserver with Master: e895f941237a48f998c154fa936cbe80 (127.13.149.194:36673)
I20260924 13:47:22.646198 16366 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.149.194@KRBTEST.COM'} at 127.13.149.194:34511
WARNING: no policy specified for kudu/127.13.149.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.149.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.149.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.195.keytab.
Entry for principal kudu/127.13.149.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.195.keytab.
I20260924 13:47:22.667760 13911 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJN9WRF/build/release/bin/kudu
/tmp/dist-test-taskJN9WRF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790257629041645-13911-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790257629041645-13911-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790257629041645-13911-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790257629041645-13911-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.149.195:0
--local_ip_for_outbound_sockets=127.13.149.195
--webserver_interface=127.13.149.195
--webserver_port=0
--tserver_master_addrs=127.13.149.254:41937
--keytab_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.195.keytab
--principal=kudu/127.13.149.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.149.212:43199
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790257629041645-13911-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790257629041645-13911-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790257629041645-13911-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 13:47:22.753042 16698 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 13:47:22.753273 16698 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 13:47:22.753304 16698 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 13:47:22.754741 16698 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 13:47:22.754822 16698 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 13:47:22.754853 16698 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.149.195
I20260924 13:47:22.757555 16698 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.149.212:43199
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790257629041645-13911-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790257629041645-13911-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.195.keytab
--principal=kudu/127.13.149.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.149.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790257629041645-13911-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.149.195
--webserver_port=0
--tserver_master_addrs=127.13.149.254:41937
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16698
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.149.195
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790257629041645-13911-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 12:29:37 UTC on ccycloud.quasar-iqzbxn.root.comops.site
build id 4670106
I20260924 13:47:22.757897 16698 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 13:47:22.758162 16698 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 13:47:22.764096 16706 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 13:47:22.764043 16698 server_base.cc:1034] running on GCE node
W20260924 13:47:22.764043 16704 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 13:47:22.764074 16703 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 13:47:22.764575 16698 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 13:47:22.764878 16698 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 13:47:22.766117 16698 hybrid_clock.cc:648] HybridClock initialized: now 1790257642766078 us; error 68 us; skew 500 ppm
Sep 24 13:47:22 dist-test-slave-sjl2 krb5kdc[16320](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257642, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.149.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 13:47:22.768893 16698 init.cc:377] Logged in from keytab as kudu/127.13.149.195@KRBTEST.COM (short username kudu)
I20260924 13:47:22.769454 16698 webserver.cc:460] Webserver started at http://127.13.149.195:34525/ using document root <none> and password file <none>
I20260924 13:47:22.769709 16698 fs_manager.cc:359] Metadata directory not provided
I20260924 13:47:22.769778 16698 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 13:47:22.769909 16698 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 13:47:22.770998 16698 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790257629041645-13911-0/minicluster-data/ts-2/data/instance:
uuid: "c1463e76ad604377b96983f0884af78f"
format_stamp: "Formatted at 2026-09-24 13:47:22 on dist-test-slave-sjl2"
server_key: "5abb5fe15467601b63a049417b49f669"
server_key_iv: "16316243f21f50e4bb3454241618d59b"
server_key_version: "encryptionkey@0"
I20260924 13:47:22.771341 16698 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790257629041645-13911-0/minicluster-data/ts-2/wal/instance:
uuid: "c1463e76ad604377b96983f0884af78f"
format_stamp: "Formatted at 2026-09-24 13:47:22 on dist-test-slave-sjl2"
server_key: "5abb5fe15467601b63a049417b49f669"
server_key_iv: "16316243f21f50e4bb3454241618d59b"
server_key_version: "encryptionkey@0"
I20260924 13:47:22.772833 16698 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260924 13:47:22.773665 16713 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 13:47:22.773869 16698 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260924 13:47:22.773958 16698 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790257629041645-13911-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790257629041645-13911-0/minicluster-data/ts-2/wal
uuid: "c1463e76ad604377b96983f0884af78f"
format_stamp: "Formatted at 2026-09-24 13:47:22 on dist-test-slave-sjl2"
server_key: "5abb5fe15467601b63a049417b49f669"
server_key_iv: "16316243f21f50e4bb3454241618d59b"
server_key_version: "encryptionkey@0"
I20260924 13:47:22.774026 16698 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790257629041645-13911-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790257629041645-13911-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790257629041645-13911-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 13:47:22.785061 16698 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 13:47:22.787634 16698 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 13:47:22.787796 16698 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 13:47:22.788031 16698 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 13:47:22.788353 16698 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 13:47:22.788404 16698 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 13:47:22.788435 16698 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 13:47:22.788460 16698 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 13:47:22.794871 16698 rpc_server.cc:307] RPC server started. Bound to: 127.13.149.195:41383
I20260924 13:47:22.794938 16828 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.149.195:41383 every 8 connection(s)
I20260924 13:47:22.795288 16698 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790257629041645-13911-0/minicluster-data/ts-2/data/info.pb
Sep 24 13:47:22 dist-test-slave-sjl2 krb5kdc[16320](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257642, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.149.195@KRBTEST.COM for kudu/127.13.149.254@KRBTEST.COM
I20260924 13:47:22.803068 13911 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJN9WRF/build/release/bin/kudu as pid 16698
I20260924 13:47:22.803169 13911 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790257629041645-13911-0/minicluster-data/ts-2/wal/instance
I20260924 13:47:22.803329 13911 external_mini_cluster.cc:1428] Setting key 709175cb7e4d4a31498a636b5163dc43
I20260924 13:47:22.805467 16406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:22.796086 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.149.195:45771 (local address 127.13.149.254:41937)
0924 13:47:22.796172 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0924 13:47:22.796179 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0924 13:47:22.796383 (+ 204us) server_negotiation.cc:407] Connection header received
0924 13:47:22.796902 (+ 519us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:47:22.796903 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:47:22.796945 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:47:22.796997 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:47:22.798075 (+ 1078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:22.798589 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:22.799832 (+ 1243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:22.800038 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:22.800108 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:47:22.802558 (+ 2450us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:47:22.802559 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:47:22.802560 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 13:47:22.802568 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:47:22.804090 (+ 1522us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:47:22.804498 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:47:22.804500 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:47:22.804500 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:47:22.804555 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:47:22.804803 (+ 248us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:47:22.804804 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:47:22.804804 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:47:22.804941 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:47:22.805005 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0924 13:47:22.805285 (+ 280us) server_negotiation.cc:299] Negotiation successful
0924 13:47:22.805373 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260924 13:47:22.805651 16831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:22.796034 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:41937 (local address 127.13.149.195:45771)
0924 13:47:22.796242 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0924 13:47:22.796300 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0924 13:47:22.796799 (+ 499us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:47:22.797068 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:47:22.797072 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:47:22.797672 (+ 600us) client_negotiation.cc:189] Negotiated authn=SASL
0924 13:47:22.797940 (+ 268us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:47:22.797947 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:47:22.798769 (+ 822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:47:22.798770 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:47:22.799673 (+ 903us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:47:22.799677 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:47:22.800117 (+ 440us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:47:22.800118 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:47:22.800238 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:47:22.800767 (+ 529us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 13:47:22.800777 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0924 13:47:22.802439 (+ 1662us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 13:47:22.804179 (+ 1740us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 13:47:22.804189 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 13:47:22.804192 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0924 13:47:22.804428 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 13:47:22.804609 (+ 181us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 13:47:22.804610 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 13:47:22.804610 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 13:47:22.804715 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 13:47:22.804994 (+ 279us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 13:47:22.805001 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 13:47:22.805159 (+ 158us) client_negotiation.cc:769] Sending connection context
0924 13:47:22.805252 (+ 93us) client_negotiation.cc:240] Negotiation successful
0924 13:47:22.805422 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":97,"threads_started":1}
I20260924 13:47:22.806156 16829 heartbeater.cc:347] Connected to a master server at 127.13.149.254:41937
I20260924 13:47:22.806245 16829 heartbeater.cc:464] Registering TS with master...
I20260924 13:47:22.806424 16829 heartbeater.cc:511] Master 127.13.149.254:41937 requested a full tablet report, sending...
I20260924 13:47:22.806802 16366 ts_manager.cc:194] Registered new tserver with Master: c1463e76ad604377b96983f0884af78f (127.13.149.195:41383)
I20260924 13:47:22.807168 16366 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.149.195@KRBTEST.COM'} at 127.13.149.195:45771
I20260924 13:47:22.815047 13911 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260924 13:47:22.825971 16406 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 13:47:22.825025 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52986 (local address 127.13.149.254:41937)
0924 13:47:22.825104 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0924 13:47:22.825107 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 13:47:22.825225 (+ 118us) server_negotiation.cc:407] Connection header received
0924 13:47:22.825301 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:47:22.825302 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:47:22.825312 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:47:22.825358 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:47:22.825909 (+ 551us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52986: BlockingRecv error: recv got EOF from 127.0.0.1:52986 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
I20260924 13:47:22.826853 13911 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJN9WRF/build/release/bin/kudu with pid 16428
I20260924 13:47:22.830631 13911 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJN9WRF/build/release/bin/kudu with pid 16563
I20260924 13:47:22.834676 13911 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJN9WRF/build/release/bin/kudu with pid 16698
I20260924 13:47:22.838532 13911 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJN9WRF/build/release/bin/kudu with pid 16334
2026-09-24T13:47:22Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (1755 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790257629041645-13911-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 13:47:22 dist-test-slave-sjl2 krb5kdc[16841](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 13:47:22 dist-test-slave-sjl2 krb5kdc[16841](info): set up 2 sockets
Sep 24 13:47:22 dist-test-slave-sjl2 krb5kdc[16841](info): commencing operation
krb5kdc: starting...
W20260924 13:47:23.801872 13911 mini_kdc.cc:121] Time spent starting KDC: real 0.944s user 0.002s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790257629041645-13911-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790257629041645-13911-0/krb5kdc/test-user.keytab.
Sep 24 13:47:23 dist-test-slave-sjl2 krb5kdc[16841](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257643, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T13:47:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T13:47:23Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.149.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.149.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.149.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.254.keytab.
Entry for principal kudu/127.13.149.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.254.keytab.
I20260924 13:47:23.927995 13911 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJN9WRF/build/release/bin/kudu
/tmp/dist-test-taskJN9WRF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790257629041645-13911-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790257629041645-13911-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790257629041645-13911-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790257629041645-13911-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.149.254:36175
--webserver_interface=127.13.149.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.254.keytab
--principal=kudu/127.13.149.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.149.212:45869
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.149.254:36175
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790257629041645-13911-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790257629041645-13911-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790257629041645-13911-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 13:47:24.010550 16854 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 13:47:24.010794 16854 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 13:47:24.010829 16854 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 13:47:24.012086 16854 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 13:47:24.012142 16854 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 13:47:24.012161 16854 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 13:47:24.012177 16854 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 13:47:24.012192 16854 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 13:47:24.013839 16854 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.149.212:45869
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790257629041645-13911-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790257629041645-13911-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.149.254:36175
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.254.keytab
--principal=kudu/127.13.149.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.149.254:36175
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790257629041645-13911-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.149.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16854
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790257629041645-13911-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 12:29:37 UTC on ccycloud.quasar-iqzbxn.root.comops.site
build id 4670106
I20260924 13:47:24.014117 16854 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 13:47:24.014334 16854 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 13:47:24.019214 16859 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 13:47:24.019233 16860 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 13:47:24.019253 16862 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 13:47:24.019345 16854 server_base.cc:1034] running on GCE node
I20260924 13:47:24.019690 16854 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 13:47:24.020005 16854 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 13:47:24.021152 16854 hybrid_clock.cc:648] HybridClock initialized: now 1790257644021122 us; error 54 us; skew 500 ppm
Sep 24 13:47:24 dist-test-slave-sjl2 krb5kdc[16841](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257644, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.149.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 13:47:24.023715 16854 init.cc:377] Logged in from keytab as kudu/127.13.149.254@KRBTEST.COM (short username kudu)
I20260924 13:47:24.024194 16854 webserver.cc:460] Webserver started at http://127.13.149.254:41493/ using document root <none> and password file <none>
I20260924 13:47:24.024431 16854 fs_manager.cc:359] Metadata directory not provided
I20260924 13:47:24.024483 16854 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 13:47:24.024614 16854 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 13:47:24.025772 16854 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790257629041645-13911-0/minicluster-data/master-0/data/instance:
uuid: "6b64b1d716d9469d8833e6e7fa657bc5"
format_stamp: "Formatted at 2026-09-24 13:47:24 on dist-test-slave-sjl2"
server_key: "82ba2a08529d2923baef6fe37bc3a437"
server_key_iv: "22b66b04c6c568a33dc8a9a49a32730f"
server_key_version: "encryptionkey@0"
I20260924 13:47:24.026113 16854 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790257629041645-13911-0/minicluster-data/master-0/wal/instance:
uuid: "6b64b1d716d9469d8833e6e7fa657bc5"
format_stamp: "Formatted at 2026-09-24 13:47:24 on dist-test-slave-sjl2"
server_key: "82ba2a08529d2923baef6fe37bc3a437"
server_key_iv: "22b66b04c6c568a33dc8a9a49a32730f"
server_key_version: "encryptionkey@0"
I20260924 13:47:24.027413 16854 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260924 13:47:24.028105 16869 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 13:47:24.028277 16854 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 13:47:24.028373 16854 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790257629041645-13911-0/minicluster-data/master-0/data,/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790257629041645-13911-0/minicluster-data/master-0/wal
uuid: "6b64b1d716d9469d8833e6e7fa657bc5"
format_stamp: "Formatted at 2026-09-24 13:47:24 on dist-test-slave-sjl2"
server_key: "82ba2a08529d2923baef6fe37bc3a437"
server_key_iv: "22b66b04c6c568a33dc8a9a49a32730f"
server_key_version: "encryptionkey@0"
I20260924 13:47:24.028447 16854 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790257629041645-13911-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790257629041645-13911-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790257629041645-13911-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 13:47:24.041676 16854 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 13:47:24.044190 16854 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 13:47:24.044356 16854 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 13:47:24.048331 16854 rpc_server.cc:307] RPC server started. Bound to: 127.13.149.254:36175
I20260924 13:47:24.048373 16921 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.149.254:36175 every 8 connection(s)
I20260924 13:47:24.048871 16854 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790257629041645-13911-0/minicluster-data/master-0/data/info.pb
I20260924 13:47:24.049646 16922 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 13:47:24.052066 16922 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6b64b1d716d9469d8833e6e7fa657bc5: Bootstrap starting.
I20260924 13:47:24.052649 16922 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6b64b1d716d9469d8833e6e7fa657bc5: Neither blocks nor log segments found. Creating new log.
I20260924 13:47:24.052978 16922 log.cc:824] T 00000000000000000000000000000000 P 6b64b1d716d9469d8833e6e7fa657bc5: Log is configured to *not* fsync() on all Append() calls
I20260924 13:47:24.053087 13911 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJN9WRF/build/release/bin/kudu as pid 16854
I20260924 13:47:24.053179 13911 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790257629041645-13911-0/minicluster-data/master-0/wal/instance
I20260924 13:47:24.053368 13911 external_mini_cluster.cc:1428] Setting key a890002278b7030990c545c951e98e1d
I20260924 13:47:24.053798 16922 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6b64b1d716d9469d8833e6e7fa657bc5: No bootstrap required, opened a new log
I20260924 13:47:24.056550 16922 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6b64b1d716d9469d8833e6e7fa657bc5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b64b1d716d9469d8833e6e7fa657bc5" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 36175 } }
I20260924 13:47:24.056708 16922 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6b64b1d716d9469d8833e6e7fa657bc5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 13:47:24.056739 16922 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6b64b1d716d9469d8833e6e7fa657bc5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6b64b1d716d9469d8833e6e7fa657bc5, State: Initialized, Role: FOLLOWER
I20260924 13:47:24.056866 16922 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6b64b1d716d9469d8833e6e7fa657bc5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b64b1d716d9469d8833e6e7fa657bc5" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 36175 } }
I20260924 13:47:24.056970 16922 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6b64b1d716d9469d8833e6e7fa657bc5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 13:47:24.057014 16922 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6b64b1d716d9469d8833e6e7fa657bc5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 13:47:24.057065 16922 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6b64b1d716d9469d8833e6e7fa657bc5 [term 0 FOLLOWER]: Advancing to term 1
I20260924 13:47:24.057819 16922 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6b64b1d716d9469d8833e6e7fa657bc5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b64b1d716d9469d8833e6e7fa657bc5" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 36175 } }
I20260924 13:47:24.057942 16922 leader_election.cc:302] T 00000000000000000000000000000000 P 6b64b1d716d9469d8833e6e7fa657bc5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6b64b1d716d9469d8833e6e7fa657bc5; no voters:
I20260924 13:47:24.058115 16922 leader_election.cc:288] T 00000000000000000000000000000000 P 6b64b1d716d9469d8833e6e7fa657bc5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 13:47:24.058159 16927 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6b64b1d716d9469d8833e6e7fa657bc5 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 13:47:24.058367 16927 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6b64b1d716d9469d8833e6e7fa657bc5 [term 1 LEADER]: Becoming Leader. State: Replica: 6b64b1d716d9469d8833e6e7fa657bc5, State: Running, Role: LEADER
I20260924 13:47:24.058389 16922 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6b64b1d716d9469d8833e6e7fa657bc5 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 13:47:24.058511 16927 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6b64b1d716d9469d8833e6e7fa657bc5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b64b1d716d9469d8833e6e7fa657bc5" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 36175 } }
I20260924 13:47:24.058966 16928 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6b64b1d716d9469d8833e6e7fa657bc5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6b64b1d716d9469d8833e6e7fa657bc5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b64b1d716d9469d8833e6e7fa657bc5" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 36175 } } }
I20260924 13:47:24.059089 16928 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6b64b1d716d9469d8833e6e7fa657bc5 [sys.catalog]: This master's current role is: LEADER
I20260924 13:47:24.058964 16929 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6b64b1d716d9469d8833e6e7fa657bc5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6b64b1d716d9469d8833e6e7fa657bc5. Latest consensus state: current_term: 1 leader_uuid: "6b64b1d716d9469d8833e6e7fa657bc5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b64b1d716d9469d8833e6e7fa657bc5" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 36175 } } }
I20260924 13:47:24.059336 16929 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6b64b1d716d9469d8833e6e7fa657bc5 [sys.catalog]: This master's current role is: LEADER
W20260924 13:47:24.060420 16942 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 6b64b1d716d9469d8833e6e7fa657bc5: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260924 13:47:24.060479 16942 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260924 13:47:24.060578 16936 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 13:47:24.060741 16936 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Sep 24 13:47:24 dist-test-slave-sjl2 krb5kdc[16841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257643, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.149.254@KRBTEST.COM
I20260924 13:47:24.062410 16936 catalog_manager.cc:1347] Generated new cluster ID: 98fdddd955d84f289fff56f87a300f30
I20260924 13:47:24.062463 16936 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 13:47:24.065649 16926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:24.053926 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45542 (local address 127.13.149.254:36175)
0924 13:47:24.054100 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0924 13:47:24.054109 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0924 13:47:24.054136 (+ 27us) server_negotiation.cc:407] Connection header received
0924 13:47:24.055105 (+ 969us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:47:24.055114 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:47:24.055171 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:47:24.055313 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:47:24.056144 (+ 831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:24.057257 (+ 1113us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:24.058272 (+ 1015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:24.058946 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:24.059050 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:47:24.061616 (+ 2566us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:47:24.061621 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:47:24.061621 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 13:47:24.061635 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:47:24.063631 (+ 1996us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:47:24.064028 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:47:24.064032 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:47:24.064032 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:47:24.064152 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:47:24.064342 (+ 190us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:47:24.064342 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:47:24.064342 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:47:24.064649 (+ 307us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:47:24.064791 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0924 13:47:24.065291 (+ 500us) server_negotiation.cc:299] Negotiation successful
0924 13:47:24.065433 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125,"thread_start_us":76,"threads_started":1}
I20260924 13:47:24.066692 13911 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260924 13:47:24.086090 16936 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 13:47:24.086664 16936 catalog_manager.cc:1504] Loading token signing keys...
I20260924 13:47:24.088013 16926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:24.083801 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45552 (local address 127.13.149.254:36175)
0924 13:47:24.083868 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0924 13:47:24.083871 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 13:47:24.083938 (+ 67us) server_negotiation.cc:407] Connection header received
0924 13:47:24.083992 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:47:24.083993 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:47:24.084005 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:47:24.084080 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:47:24.084537 (+ 457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:24.085372 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:24.086320 (+ 948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:24.086658 (+ 338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:24.086733 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:47:24.087510 (+ 777us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:47:24.087512 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:47:24.087512 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 13:47:24.087745 (+ 233us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:47:24.087765 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 13:47:24.087773 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:47:24.087835 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0924 13:47:24.087880 (+ 45us) server_negotiation.cc:299] Negotiation successful
0924 13:47:24.087934 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260924 13:47:24.091046 16936 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6b64b1d716d9469d8833e6e7fa657bc5: Generated new TSK 0
I20260924 13:47:24.091259 16936 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260924 13:47:24.094444 13911 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJN9WRF/build/release/bin/kudu with pid 16854
2026-09-24T13:47:24Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1253 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790257629041645-13911-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 13:47:24 dist-test-slave-sjl2 krb5kdc[16955](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 13:47:24 dist-test-slave-sjl2 krb5kdc[16955](info): set up 2 sockets
Sep 24 13:47:24 dist-test-slave-sjl2 krb5kdc[16955](info): commencing operation
krb5kdc: starting...
W20260924 13:47:25.049738 13911 mini_kdc.cc:121] Time spent starting KDC: real 0.938s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790257629041645-13911-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790257629041645-13911-0/krb5kdc/test-user.keytab.
Sep 24 13:47:25 dist-test-slave-sjl2 krb5kdc[16955](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257645, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T13:47:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T13:47:25Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.149.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.149.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.149.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.254.keytab.
Entry for principal kudu/127.13.149.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.254.keytab.
I20260924 13:47:25.171846 13911 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJN9WRF/build/release/bin/kudu
/tmp/dist-test-taskJN9WRF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790257629041645-13911-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790257629041645-13911-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790257629041645-13911-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790257629041645-13911-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.149.254:40855
--webserver_interface=127.13.149.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.254.keytab
--principal=kudu/127.13.149.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.149.212:40505
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.149.254:40855
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790257629041645-13911-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790257629041645-13911-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790257629041645-13911-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 13:47:25.256527 16969 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 13:47:25.256788 16969 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 13:47:25.256824 16969 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 13:47:25.258152 16969 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 13:47:25.258211 16969 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 13:47:25.258241 16969 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 13:47:25.258256 16969 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 13:47:25.258273 16969 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 13:47:25.260003 16969 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.149.212:40505
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790257629041645-13911-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790257629041645-13911-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.149.254:40855
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.254.keytab
--principal=kudu/127.13.149.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.149.254:40855
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790257629041645-13911-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.149.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16969
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790257629041645-13911-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 12:29:37 UTC on ccycloud.quasar-iqzbxn.root.comops.site
build id 4670106
I20260924 13:47:25.260308 16969 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 13:47:25.260541 16969 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 13:47:25.265973 16975 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 13:47:25.266008 16977 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 13:47:25.265976 16974 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 13:47:25.266158 16969 server_base.cc:1034] running on GCE node
I20260924 13:47:25.266405 16969 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 13:47:25.266709 16969 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 13:47:25.267874 16969 hybrid_clock.cc:648] HybridClock initialized: now 1790257645267850 us; error 38 us; skew 500 ppm
Sep 24 13:47:25 dist-test-slave-sjl2 krb5kdc[16955](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257645, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.149.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 13:47:25.270476 16969 init.cc:377] Logged in from keytab as kudu/127.13.149.254@KRBTEST.COM (short username kudu)
I20260924 13:47:25.270964 16969 webserver.cc:460] Webserver started at http://127.13.149.254:34459/ using document root <none> and password file <none>
I20260924 13:47:25.271190 16969 fs_manager.cc:359] Metadata directory not provided
I20260924 13:47:25.271241 16969 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 13:47:25.271377 16969 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 13:47:25.272481 16969 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790257629041645-13911-0/minicluster-data/master-0/data/instance:
uuid: "6410c92d82b8410ba1a971d8b4aa360d"
format_stamp: "Formatted at 2026-09-24 13:47:25 on dist-test-slave-sjl2"
server_key: "72913a8fe564b469b9d722004f118c84"
server_key_iv: "99e3e5213720ff17c204d184a86d6818"
server_key_version: "encryptionkey@0"
I20260924 13:47:25.272831 16969 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790257629041645-13911-0/minicluster-data/master-0/wal/instance:
uuid: "6410c92d82b8410ba1a971d8b4aa360d"
format_stamp: "Formatted at 2026-09-24 13:47:25 on dist-test-slave-sjl2"
server_key: "72913a8fe564b469b9d722004f118c84"
server_key_iv: "99e3e5213720ff17c204d184a86d6818"
server_key_version: "encryptionkey@0"
I20260924 13:47:25.274185 16969 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260924 13:47:25.274979 16984 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 13:47:25.275161 16969 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260924 13:47:25.275249 16969 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790257629041645-13911-0/minicluster-data/master-0/data,/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790257629041645-13911-0/minicluster-data/master-0/wal
uuid: "6410c92d82b8410ba1a971d8b4aa360d"
format_stamp: "Formatted at 2026-09-24 13:47:25 on dist-test-slave-sjl2"
server_key: "72913a8fe564b469b9d722004f118c84"
server_key_iv: "99e3e5213720ff17c204d184a86d6818"
server_key_version: "encryptionkey@0"
I20260924 13:47:25.275321 16969 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790257629041645-13911-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790257629041645-13911-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790257629041645-13911-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 13:47:25.284224 16969 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 13:47:25.286499 16969 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 13:47:25.286650 16969 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 13:47:25.290654 16969 rpc_server.cc:307] RPC server started. Bound to: 127.13.149.254:40855
I20260924 13:47:25.290705 17036 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.149.254:40855 every 8 connection(s)
I20260924 13:47:25.291060 16969 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790257629041645-13911-0/minicluster-data/master-0/data/info.pb
I20260924 13:47:25.292100 17037 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 13:47:25.294546 17037 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6410c92d82b8410ba1a971d8b4aa360d: Bootstrap starting.
I20260924 13:47:25.295163 17037 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6410c92d82b8410ba1a971d8b4aa360d: Neither blocks nor log segments found. Creating new log.
I20260924 13:47:25.295428 17037 log.cc:824] T 00000000000000000000000000000000 P 6410c92d82b8410ba1a971d8b4aa360d: Log is configured to *not* fsync() on all Append() calls
I20260924 13:47:25.296227 17037 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6410c92d82b8410ba1a971d8b4aa360d: No bootstrap required, opened a new log
I20260924 13:47:25.297566 13911 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJN9WRF/build/release/bin/kudu as pid 16969
I20260924 13:47:25.297683 13911 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790257629041645-13911-0/minicluster-data/master-0/wal/instance
I20260924 13:47:25.297842 13911 external_mini_cluster.cc:1428] Setting key 58bb10a5cf4e9e4393fd082a653ba6ae
I20260924 13:47:25.298115 17037 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6410c92d82b8410ba1a971d8b4aa360d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6410c92d82b8410ba1a971d8b4aa360d" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 40855 } }
I20260924 13:47:25.298363 17037 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6410c92d82b8410ba1a971d8b4aa360d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 13:47:25.298442 17037 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6410c92d82b8410ba1a971d8b4aa360d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6410c92d82b8410ba1a971d8b4aa360d, State: Initialized, Role: FOLLOWER
I20260924 13:47:25.298585 17037 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6410c92d82b8410ba1a971d8b4aa360d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6410c92d82b8410ba1a971d8b4aa360d" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 40855 } }
I20260924 13:47:25.298669 17037 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6410c92d82b8410ba1a971d8b4aa360d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 13:47:25.298692 17037 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6410c92d82b8410ba1a971d8b4aa360d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 13:47:25.298751 17037 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6410c92d82b8410ba1a971d8b4aa360d [term 0 FOLLOWER]: Advancing to term 1
I20260924 13:47:25.299500 17037 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6410c92d82b8410ba1a971d8b4aa360d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6410c92d82b8410ba1a971d8b4aa360d" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 40855 } }
I20260924 13:47:25.299621 17037 leader_election.cc:302] T 00000000000000000000000000000000 P 6410c92d82b8410ba1a971d8b4aa360d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6410c92d82b8410ba1a971d8b4aa360d; no voters:
I20260924 13:47:25.299809 17037 leader_election.cc:288] T 00000000000000000000000000000000 P 6410c92d82b8410ba1a971d8b4aa360d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 13:47:25.299845 17042 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6410c92d82b8410ba1a971d8b4aa360d [term 1 FOLLOWER]: Leader election won for term 1
I20260924 13:47:25.299988 17042 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6410c92d82b8410ba1a971d8b4aa360d [term 1 LEADER]: Becoming Leader. State: Replica: 6410c92d82b8410ba1a971d8b4aa360d, State: Running, Role: LEADER
I20260924 13:47:25.300086 17037 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6410c92d82b8410ba1a971d8b4aa360d [sys.catalog]: configured and running, proceeding with master startup.
I20260924 13:47:25.300137 17042 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6410c92d82b8410ba1a971d8b4aa360d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6410c92d82b8410ba1a971d8b4aa360d" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 40855 } }
I20260924 13:47:25.300544 17042 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6410c92d82b8410ba1a971d8b4aa360d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6410c92d82b8410ba1a971d8b4aa360d. Latest consensus state: current_term: 1 leader_uuid: "6410c92d82b8410ba1a971d8b4aa360d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6410c92d82b8410ba1a971d8b4aa360d" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 40855 } } }
I20260924 13:47:25.300607 17043 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6410c92d82b8410ba1a971d8b4aa360d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6410c92d82b8410ba1a971d8b4aa360d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6410c92d82b8410ba1a971d8b4aa360d" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 40855 } } }
I20260924 13:47:25.300678 17042 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6410c92d82b8410ba1a971d8b4aa360d [sys.catalog]: This master's current role is: LEADER
I20260924 13:47:25.300706 17043 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6410c92d82b8410ba1a971d8b4aa360d [sys.catalog]: This master's current role is: LEADER
I20260924 13:47:25.300901 17051 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 13:47:25.301714 17051 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 13:47:25.303469 17051 catalog_manager.cc:1347] Generated new cluster ID: 7978d05e1767438dbb786093a19a7f3a
I20260924 13:47:25.303524 17051 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 24 13:47:25 dist-test-slave-sjl2 krb5kdc[16955](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257645, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.149.254@KRBTEST.COM
I20260924 13:47:25.310070 17041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:25.298425 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43776 (local address 127.13.149.254:40855)
0924 13:47:25.298641 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0924 13:47:25.298651 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0924 13:47:25.298686 (+ 35us) server_negotiation.cc:407] Connection header received
0924 13:47:25.298879 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:47:25.298886 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:47:25.298934 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:47:25.299050 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:47:25.299809 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:25.301374 (+ 1565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:25.302528 (+ 1154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:25.303041 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:25.303178 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:47:25.306043 (+ 2865us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:47:25.306048 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:47:25.306048 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 13:47:25.306064 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:47:25.308106 (+ 2042us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:47:25.308528 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:47:25.308532 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:47:25.308532 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:47:25.308677 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:47:25.308859 (+ 182us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:47:25.308860 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:47:25.308861 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:47:25.309269 (+ 408us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:47:25.309460 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0924 13:47:25.309538 (+ 78us) server_negotiation.cc:299] Negotiation successful
0924 13:47:25.309828 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":158,"thread_start_us":125,"threads_started":1}
I20260924 13:47:25.325807 17051 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 13:47:25.326650 17051 catalog_manager.cc:1504] Loading token signing keys...
I20260924 13:47:25.331086 17051 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6410c92d82b8410ba1a971d8b4aa360d: Generated new TSK 0
I20260924 13:47:25.331321 17051 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.13.149.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.149.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.149.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.193.keytab.
Entry for principal kudu/127.13.149.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.193.keytab.
I20260924 13:47:25.359457 13911 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJN9WRF/build/release/bin/kudu
/tmp/dist-test-taskJN9WRF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790257629041645-13911-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790257629041645-13911-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790257629041645-13911-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790257629041645-13911-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.149.193:0
--local_ip_for_outbound_sockets=127.13.149.193
--webserver_interface=127.13.149.193
--webserver_port=0
--tserver_master_addrs=127.13.149.254:40855
--keytab_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.193.keytab
--principal=kudu/127.13.149.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.149.212:40505
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790257629041645-13911-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790257629041645-13911-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790257629041645-13911-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 13:47:25.451028 17063 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 13:47:25.451314 17063 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 13:47:25.451352 17063 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 13:47:25.453245 17063 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 13:47:25.453316 17063 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 13:47:25.453365 17063 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.149.193
I20260924 13:47:25.455914 17063 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.149.212:40505
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790257629041645-13911-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790257629041645-13911-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.193.keytab
--principal=kudu/127.13.149.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.149.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790257629041645-13911-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.149.193
--webserver_port=0
--tserver_master_addrs=127.13.149.254:40855
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17063
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.149.193
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790257629041645-13911-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 12:29:37 UTC on ccycloud.quasar-iqzbxn.root.comops.site
build id 4670106
I20260924 13:47:25.456369 17063 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 13:47:25.456761 17063 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 13:47:25.462240 17068 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 13:47:25.462245 17069 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 13:47:25.462261 17063 server_base.cc:1034] running on GCE node
W20260924 13:47:25.462244 17071 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 13:47:25.462615 17063 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 13:47:25.462888 17063 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 13:47:25.464076 17063 hybrid_clock.cc:648] HybridClock initialized: now 1790257645464060 us; error 30 us; skew 500 ppm
Sep 24 13:47:25 dist-test-slave-sjl2 krb5kdc[16955](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257645, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.149.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 13:47:25.470856 17063 init.cc:377] Logged in from keytab as kudu/127.13.149.193@KRBTEST.COM (short username kudu)
I20260924 13:47:25.471386 17063 webserver.cc:460] Webserver started at http://127.13.149.193:45723/ using document root <none> and password file <none>
I20260924 13:47:25.471611 17063 fs_manager.cc:359] Metadata directory not provided
I20260924 13:47:25.471655 17063 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 13:47:25.471756 17063 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 13:47:25.473356 17063 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790257629041645-13911-0/minicluster-data/ts-0/data/instance:
uuid: "f5e75c54f4664eb983de55549693fd23"
format_stamp: "Formatted at 2026-09-24 13:47:25 on dist-test-slave-sjl2"
server_key: "60807e87923b2dad5bb352844b1b6ad7"
server_key_iv: "398a18d5c615f36815b77d6148a81d99"
server_key_version: "encryptionkey@0"
I20260924 13:47:25.473819 17063 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790257629041645-13911-0/minicluster-data/ts-0/wal/instance:
uuid: "f5e75c54f4664eb983de55549693fd23"
format_stamp: "Formatted at 2026-09-24 13:47:25 on dist-test-slave-sjl2"
server_key: "60807e87923b2dad5bb352844b1b6ad7"
server_key_iv: "398a18d5c615f36815b77d6148a81d99"
server_key_version: "encryptionkey@0"
I20260924 13:47:25.475705 17063 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20260924 13:47:25.476903 17078 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 13:47:25.477176 17063 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260924 13:47:25.477293 17063 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790257629041645-13911-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790257629041645-13911-0/minicluster-data/ts-0/wal
uuid: "f5e75c54f4664eb983de55549693fd23"
format_stamp: "Formatted at 2026-09-24 13:47:25 on dist-test-slave-sjl2"
server_key: "60807e87923b2dad5bb352844b1b6ad7"
server_key_iv: "398a18d5c615f36815b77d6148a81d99"
server_key_version: "encryptionkey@0"
I20260924 13:47:25.477397 17063 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790257629041645-13911-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790257629041645-13911-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790257629041645-13911-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 13:47:25.501763 17063 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 13:47:25.504496 17063 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 13:47:25.504667 17063 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 13:47:25.504964 17063 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 13:47:25.505337 17063 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 13:47:25.505381 17063 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 13:47:25.505417 17063 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 13:47:25.505439 17063 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 13:47:25.511509 17063 rpc_server.cc:307] RPC server started. Bound to: 127.13.149.193:35027
I20260924 13:47:25.511569 17192 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.149.193:35027 every 8 connection(s)
I20260924 13:47:25.511920 17063 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790257629041645-13911-0/minicluster-data/ts-0/data/info.pb
I20260924 13:47:25.514892 13911 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJN9WRF/build/release/bin/kudu as pid 17063
I20260924 13:47:25.514994 13911 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790257629041645-13911-0/minicluster-data/ts-0/wal/instance
I20260924 13:47:25.515167 13911 external_mini_cluster.cc:1428] Setting key 4aaa54adb8110787719978ae613140fd
Sep 24 13:47:25 dist-test-slave-sjl2 krb5kdc[16955](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257645, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.149.193@KRBTEST.COM for kudu/127.13.149.254@KRBTEST.COM
I20260924 13:47:25.522272 17041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:25.512687 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.149.193:52721 (local address 127.13.149.254:40855)
0924 13:47:25.512754 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0924 13:47:25.512756 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:47:25.512984 (+ 228us) server_negotiation.cc:407] Connection header received
0924 13:47:25.513560 (+ 576us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:47:25.513561 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:47:25.513576 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:47:25.513626 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:47:25.514686 (+ 1060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:25.515285 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:25.516697 (+ 1412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:25.516904 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:25.516995 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:47:25.519579 (+ 2584us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:47:25.519580 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:47:25.519581 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 13:47:25.519589 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:47:25.520874 (+ 1285us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:47:25.521311 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:47:25.521313 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:47:25.521313 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:47:25.521373 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:47:25.521610 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:47:25.521611 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:47:25.521611 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:47:25.521736 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:47:25.521794 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0924 13:47:25.522083 (+ 289us) server_negotiation.cc:299] Negotiation successful
0924 13:47:25.522176 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260924 13:47:25.522464 17195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:25.512659 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:40855 (local address 127.13.149.193:52721)
0924 13:47:25.512833 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0924 13:47:25.512878 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0924 13:47:25.513461 (+ 583us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:47:25.513714 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:47:25.513718 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:47:25.514357 (+ 639us) client_negotiation.cc:189] Negotiated authn=SASL
0924 13:47:25.514575 (+ 218us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:47:25.514580 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:47:25.515409 (+ 829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:47:25.515411 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:47:25.516607 (+ 1196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:47:25.516611 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:47:25.517007 (+ 396us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:47:25.517008 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:47:25.517171 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:47:25.517848 (+ 677us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 13:47:25.517857 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0924 13:47:25.519475 (+ 1618us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 13:47:25.520990 (+ 1515us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 13:47:25.520999 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 13:47:25.521002 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0924 13:47:25.521236 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 13:47:25.521428 (+ 192us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 13:47:25.521429 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 13:47:25.521429 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 13:47:25.521552 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 13:47:25.521791 (+ 239us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 13:47:25.521798 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 13:47:25.521952 (+ 154us) client_negotiation.cc:769] Sending connection context
0924 13:47:25.522049 (+ 97us) client_negotiation.cc:240] Negotiation successful
0924 13:47:25.522215 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":86,"threads_started":1}
I20260924 13:47:25.522930 17193 heartbeater.cc:347] Connected to a master server at 127.13.149.254:40855
I20260924 13:47:25.523046 17193 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.13.149.194@KRBTEST.COM; defaulting to no policy
I20260924 13:47:25.523264 17193 heartbeater.cc:511] Master 127.13.149.254:40855 requested a full tablet report, sending...
I20260924 13:47:25.523847 17001 ts_manager.cc:194] Registered new tserver with Master: f5e75c54f4664eb983de55549693fd23 (127.13.149.193:35027)
I20260924 13:47:25.524479 17001 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.149.193@KRBTEST.COM'} at 127.13.149.193:52721
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.149.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.149.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.194.keytab.
Entry for principal kudu/127.13.149.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.194.keytab.
W20260924 13:47:26.187703 13911 mini_kdc.cc:262] Time spent creating service keytab for kudu/127.13.149.194: real 0.672s user 0.000s sys 0.003s
I20260924 13:47:26.188062 13911 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJN9WRF/build/release/bin/kudu
/tmp/dist-test-taskJN9WRF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790257629041645-13911-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790257629041645-13911-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790257629041645-13911-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790257629041645-13911-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.149.194:0
--local_ip_for_outbound_sockets=127.13.149.194
--webserver_interface=127.13.149.194
--webserver_port=0
--tserver_master_addrs=127.13.149.254:40855
--keytab_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.194.keytab
--principal=kudu/127.13.149.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.149.212:40505
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790257629041645-13911-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790257629041645-13911-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790257629041645-13911-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 13:47:26.272365 17198 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 13:47:26.272593 17198 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 13:47:26.272619 17198 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 13:47:26.273919 17198 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 13:47:26.273976 17198 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 13:47:26.274008 17198 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.149.194
I20260924 13:47:26.275592 17198 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.149.212:40505
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790257629041645-13911-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790257629041645-13911-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.194.keytab
--principal=kudu/127.13.149.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.149.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790257629041645-13911-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.149.194
--webserver_port=0
--tserver_master_addrs=127.13.149.254:40855
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17198
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.149.194
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790257629041645-13911-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 12:29:37 UTC on ccycloud.quasar-iqzbxn.root.comops.site
build id 4670106
I20260924 13:47:26.275876 17198 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 13:47:26.276103 17198 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 13:47:26.281368 17204 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 13:47:26.281371 17206 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 13:47:26.281371 17203 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 13:47:26.281389 17198 server_base.cc:1034] running on GCE node
I20260924 13:47:26.281836 17198 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 13:47:26.282099 17198 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 13:47:26.283250 17198 hybrid_clock.cc:648] HybridClock initialized: now 1790257646283238 us; error 34 us; skew 500 ppm
Sep 24 13:47:26 dist-test-slave-sjl2 krb5kdc[16955](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257646, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.149.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 13:47:26.285907 17198 init.cc:377] Logged in from keytab as kudu/127.13.149.194@KRBTEST.COM (short username kudu)
I20260924 13:47:26.286370 17198 webserver.cc:460] Webserver started at http://127.13.149.194:40645/ using document root <none> and password file <none>
I20260924 13:47:26.286597 17198 fs_manager.cc:359] Metadata directory not provided
I20260924 13:47:26.286650 17198 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 13:47:26.286795 17198 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 13:47:26.287925 17198 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790257629041645-13911-0/minicluster-data/ts-1/data/instance:
uuid: "a53f8c83754c4a3993c2db3167ddb1b0"
format_stamp: "Formatted at 2026-09-24 13:47:26 on dist-test-slave-sjl2"
server_key: "5f480570d9996a0fcd57c91b2a15d773"
server_key_iv: "91087782276d026fe8f558152e31d1c4"
server_key_version: "encryptionkey@0"
I20260924 13:47:26.288303 17198 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790257629041645-13911-0/minicluster-data/ts-1/wal/instance:
uuid: "a53f8c83754c4a3993c2db3167ddb1b0"
format_stamp: "Formatted at 2026-09-24 13:47:26 on dist-test-slave-sjl2"
server_key: "5f480570d9996a0fcd57c91b2a15d773"
server_key_iv: "91087782276d026fe8f558152e31d1c4"
server_key_version: "encryptionkey@0"
I20260924 13:47:26.289779 17198 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260924 13:47:26.290566 17213 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 13:47:26.290823 17198 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260924 13:47:26.290920 17198 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790257629041645-13911-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790257629041645-13911-0/minicluster-data/ts-1/wal
uuid: "a53f8c83754c4a3993c2db3167ddb1b0"
format_stamp: "Formatted at 2026-09-24 13:47:26 on dist-test-slave-sjl2"
server_key: "5f480570d9996a0fcd57c91b2a15d773"
server_key_iv: "91087782276d026fe8f558152e31d1c4"
server_key_version: "encryptionkey@0"
I20260924 13:47:26.291002 17198 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790257629041645-13911-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790257629041645-13911-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790257629041645-13911-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 13:47:26.312099 17198 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 13:47:26.314566 17198 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 13:47:26.314762 17198 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 13:47:26.314976 17198 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 13:47:26.315312 17198 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 13:47:26.315346 17198 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 13:47:26.315371 17198 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 13:47:26.315393 17198 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 13:47:26.321237 17198 rpc_server.cc:307] RPC server started. Bound to: 127.13.149.194:37343
I20260924 13:47:26.321287 17328 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.149.194:37343 every 8 connection(s)
I20260924 13:47:26.321620 17198 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790257629041645-13911-0/minicluster-data/ts-1/data/info.pb
I20260924 13:47:26.323146 13911 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJN9WRF/build/release/bin/kudu as pid 17198
I20260924 13:47:26.323263 13911 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790257629041645-13911-0/minicluster-data/ts-1/wal/instance
I20260924 13:47:26.323441 13911 external_mini_cluster.cc:1428] Setting key 75622f5af3b34025e77de331003ffd59
Sep 24 13:47:26 dist-test-slave-sjl2 krb5kdc[16955](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257646, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.149.194@KRBTEST.COM for kudu/127.13.149.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.13.149.195@KRBTEST.COM; defaulting to no policy
I20260924 13:47:26.333022 17332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:26.322367 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.149.194:45123 (local address 127.13.149.254:40855)
0924 13:47:26.322481 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0924 13:47:26.322483 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:47:26.322659 (+ 176us) server_negotiation.cc:407] Connection header received
0924 13:47:26.323145 (+ 486us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:47:26.323147 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:47:26.323163 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:47:26.323266 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:47:26.324363 (+ 1097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:26.325111 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:26.326620 (+ 1509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:26.326837 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:26.326914 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:47:26.329667 (+ 2753us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:47:26.329669 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:47:26.329670 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 13:47:26.329679 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:47:26.331387 (+ 1708us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:47:26.332035 (+ 648us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:47:26.332037 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:47:26.332038 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:47:26.332103 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:47:26.332343 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:47:26.332343 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:47:26.332343 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:47:26.332459 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:47:26.332507 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0924 13:47:26.332789 (+ 282us) server_negotiation.cc:299] Negotiation successful
0924 13:47:26.332908 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91,"thread_start_us":58,"threads_started":1}
I20260924 13:47:26.333128 17331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:26.322336 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:40855 (local address 127.13.149.194:45123)
0924 13:47:26.322498 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0924 13:47:26.322542 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0924 13:47:26.323035 (+ 493us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:47:26.323368 (+ 333us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:47:26.323372 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:47:26.324000 (+ 628us) client_negotiation.cc:189] Negotiated authn=SASL
0924 13:47:26.324266 (+ 266us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:47:26.324272 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:47:26.325216 (+ 944us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:47:26.325217 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:47:26.326502 (+ 1285us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:47:26.326505 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:47:26.326953 (+ 448us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:47:26.326955 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:47:26.327102 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:47:26.327685 (+ 583us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 13:47:26.327694 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0924 13:47:26.329545 (+ 1851us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 13:47:26.331558 (+ 2013us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 13:47:26.331571 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 13:47:26.331574 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0924 13:47:26.331953 (+ 379us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 13:47:26.332164 (+ 211us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 13:47:26.332164 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 13:47:26.332165 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 13:47:26.332296 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 13:47:26.332506 (+ 210us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 13:47:26.332510 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 13:47:26.332645 (+ 135us) client_negotiation.cc:769] Sending connection context
0924 13:47:26.332730 (+ 85us) client_negotiation.cc:240] Negotiation successful
0924 13:47:26.332867 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":66,"threads_started":1}
I20260924 13:47:26.333585 17329 heartbeater.cc:347] Connected to a master server at 127.13.149.254:40855
I20260924 13:47:26.333691 17329 heartbeater.cc:464] Registering TS with master...
I20260924 13:47:26.333890 17329 heartbeater.cc:511] Master 127.13.149.254:40855 requested a full tablet report, sending...
I20260924 13:47:26.334301 17001 ts_manager.cc:194] Registered new tserver with Master: a53f8c83754c4a3993c2db3167ddb1b0 (127.13.149.194:37343)
I20260924 13:47:26.334654 17001 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.149.194@KRBTEST.COM'} at 127.13.149.194:45123
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.149.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.149.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.195.keytab.
Entry for principal kudu/127.13.149.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.195.keytab.
I20260924 13:47:26.352043 13911 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJN9WRF/build/release/bin/kudu
/tmp/dist-test-taskJN9WRF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790257629041645-13911-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790257629041645-13911-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790257629041645-13911-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790257629041645-13911-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.149.195:0
--local_ip_for_outbound_sockets=127.13.149.195
--webserver_interface=127.13.149.195
--webserver_port=0
--tserver_master_addrs=127.13.149.254:40855
--keytab_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.195.keytab
--principal=kudu/127.13.149.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.149.212:40505
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790257629041645-13911-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790257629041645-13911-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790257629041645-13911-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 13:47:26.434347 17335 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 13:47:26.434592 17335 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 13:47:26.434628 17335 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 13:47:26.435967 17335 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 13:47:26.436036 17335 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 13:47:26.436079 17335 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.149.195
I20260924 13:47:26.438055 17335 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.149.212:40505
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790257629041645-13911-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790257629041645-13911-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.195.keytab
--principal=kudu/127.13.149.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.149.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790257629041645-13911-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.149.195
--webserver_port=0
--tserver_master_addrs=127.13.149.254:40855
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17335
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.149.195
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790257629041645-13911-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 12:29:37 UTC on ccycloud.quasar-iqzbxn.root.comops.site
build id 4670106
I20260924 13:47:26.438426 17335 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 13:47:26.438668 17335 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260924 13:47:26.444036 17335 server_base.cc:1034] running on GCE node
W20260924 13:47:26.443979 17344 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 13:47:26.443979 17341 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 13:47:26.443979 17342 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 13:47:26.444511 17335 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 13:47:26.444753 17335 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 13:47:26.445894 17335 hybrid_clock.cc:648] HybridClock initialized: now 1790257646445875 us; error 42 us; skew 500 ppm
Sep 24 13:47:26 dist-test-slave-sjl2 krb5kdc[16955](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257646, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.149.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 13:47:26.448559 17335 init.cc:377] Logged in from keytab as kudu/127.13.149.195@KRBTEST.COM (short username kudu)
I20260924 13:47:26.449079 17335 webserver.cc:460] Webserver started at http://127.13.149.195:39197/ using document root <none> and password file <none>
I20260924 13:47:26.449322 17335 fs_manager.cc:359] Metadata directory not provided
I20260924 13:47:26.449373 17335 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 13:47:26.449503 17335 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 13:47:26.450580 17335 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790257629041645-13911-0/minicluster-data/ts-2/data/instance:
uuid: "960e367430ac4561b88b2bbc9d20ad53"
format_stamp: "Formatted at 2026-09-24 13:47:26 on dist-test-slave-sjl2"
server_key: "831eb990e7eb0237bb1e5428cc29b608"
server_key_iv: "65b5ce95333ac4d1501958333b33a37d"
server_key_version: "encryptionkey@0"
I20260924 13:47:26.450948 17335 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790257629041645-13911-0/minicluster-data/ts-2/wal/instance:
uuid: "960e367430ac4561b88b2bbc9d20ad53"
format_stamp: "Formatted at 2026-09-24 13:47:26 on dist-test-slave-sjl2"
server_key: "831eb990e7eb0237bb1e5428cc29b608"
server_key_iv: "65b5ce95333ac4d1501958333b33a37d"
server_key_version: "encryptionkey@0"
I20260924 13:47:26.452615 17335 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260924 13:47:26.453760 17351 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 13:47:26.454046 17335 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260924 13:47:26.454157 17335 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790257629041645-13911-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790257629041645-13911-0/minicluster-data/ts-2/wal
uuid: "960e367430ac4561b88b2bbc9d20ad53"
format_stamp: "Formatted at 2026-09-24 13:47:26 on dist-test-slave-sjl2"
server_key: "831eb990e7eb0237bb1e5428cc29b608"
server_key_iv: "65b5ce95333ac4d1501958333b33a37d"
server_key_version: "encryptionkey@0"
I20260924 13:47:26.454267 17335 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790257629041645-13911-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790257629041645-13911-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790257629041645-13911-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 13:47:26.469945 17335 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 13:47:26.472044 17335 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 13:47:26.472190 17335 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 13:47:26.472417 17335 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 13:47:26.472700 17335 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 13:47:26.472739 17335 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 13:47:26.472764 17335 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 13:47:26.472779 17335 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 13:47:26.478428 17335 rpc_server.cc:307] RPC server started. Bound to: 127.13.149.195:43295
I20260924 13:47:26.478467 17467 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.149.195:43295 every 8 connection(s)
I20260924 13:47:26.478812 17335 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790257629041645-13911-0/minicluster-data/ts-2/data/info.pb
Sep 24 13:47:26 dist-test-slave-sjl2 krb5kdc[16955](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257646, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.149.195@KRBTEST.COM for kudu/127.13.149.254@KRBTEST.COM
I20260924 13:47:26.487397 13911 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJN9WRF/build/release/bin/kudu as pid 17335
I20260924 13:47:26.487498 13911 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790257629041645-13911-0/minicluster-data/ts-2/wal/instance
I20260924 13:47:26.487703 13911 external_mini_cluster.cc:1428] Setting key a93493bacdc1281d91347e02e6039c22
I20260924 13:47:26.488795 17332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:26.479585 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.149.195:57485 (local address 127.13.149.254:40855)
0924 13:47:26.479632 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0924 13:47:26.479634 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:47:26.479871 (+ 237us) server_negotiation.cc:407] Connection header received
0924 13:47:26.480434 (+ 563us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:47:26.480435 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:47:26.480448 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:47:26.480502 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:47:26.481531 (+ 1029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:26.482041 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:26.483041 (+ 1000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:26.483248 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:26.483305 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:47:26.485742 (+ 2437us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:47:26.485744 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:47:26.485745 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 13:47:26.485753 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:47:26.487089 (+ 1336us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:47:26.487614 (+ 525us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:47:26.487616 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:47:26.487617 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:47:26.487725 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:47:26.488005 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:47:26.488005 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:47:26.488006 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:47:26.488166 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:47:26.488230 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0924 13:47:26.488615 (+ 385us) server_negotiation.cc:299] Negotiation successful
0924 13:47:26.488728 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260924 13:47:26.489034 17470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:26.479592 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:40855 (local address 127.13.149.195:57485)
0924 13:47:26.479751 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0924 13:47:26.479788 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0924 13:47:26.480327 (+ 539us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:47:26.480575 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:47:26.480580 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:47:26.481227 (+ 647us) client_negotiation.cc:189] Negotiated authn=SASL
0924 13:47:26.481430 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:47:26.481435 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:47:26.482128 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:47:26.482129 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:47:26.482948 (+ 819us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:47:26.482951 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:47:26.483330 (+ 379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:47:26.483332 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:47:26.483458 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:47:26.484015 (+ 557us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 13:47:26.484025 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0924 13:47:26.485624 (+ 1599us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 13:47:26.487242 (+ 1618us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 13:47:26.487250 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 13:47:26.487252 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 13:47:26.487495 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 13:47:26.487808 (+ 313us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 13:47:26.487809 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 13:47:26.487809 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 13:47:26.487933 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 13:47:26.488266 (+ 333us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 13:47:26.488281 (+ 15us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 13:47:26.488469 (+ 188us) client_negotiation.cc:769] Sending connection context
0924 13:47:26.488583 (+ 114us) client_negotiation.cc:240] Negotiation successful
0924 13:47:26.488772 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":79,"threads_started":1}
I20260924 13:47:26.489480 17468 heartbeater.cc:347] Connected to a master server at 127.13.149.254:40855
I20260924 13:47:26.489576 17468 heartbeater.cc:464] Registering TS with master...
I20260924 13:47:26.489739 17468 heartbeater.cc:511] Master 127.13.149.254:40855 requested a full tablet report, sending...
I20260924 13:47:26.490031 17001 ts_manager.cc:194] Registered new tserver with Master: 960e367430ac4561b88b2bbc9d20ad53 (127.13.149.195:43295)
I20260924 13:47:26.490409 17001 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.149.195@KRBTEST.COM'} at 127.13.149.195:57485
I20260924 13:47:26.499724 13911 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 24 13:47:26 dist-test-slave-sjl2 krb5kdc[16955](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257646, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Sep 24 13:47:26 dist-test-slave-sjl2 krb5kdc[16955](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257646, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.13.149.254@KRBTEST.COM
I20260924 13:47:26.523533 17332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:26.515911 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43780 (local address 127.13.149.254:40855)
0924 13:47:26.515966 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0924 13:47:26.515968 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:47:26.516038 (+ 70us) server_negotiation.cc:407] Connection header received
0924 13:47:26.516094 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:47:26.516095 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:47:26.516105 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:47:26.516175 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:47:26.516787 (+ 612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:26.517393 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:26.518211 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:26.518418 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:26.518490 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:47:26.521072 (+ 2582us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:47:26.521074 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:47:26.521074 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 13:47:26.521083 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:47:26.522425 (+ 1342us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:47:26.522793 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:47:26.522794 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:47:26.522794 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:47:26.522848 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:47:26.523063 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:47:26.523064 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:47:26.523064 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:47:26.523223 (+ 159us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:47:26.523301 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0924 13:47:26.523394 (+ 93us) server_negotiation.cc:299] Negotiation successful
0924 13:47:26.523464 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
W20260924 13:47:26.523798 17001 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:43780
I20260924 13:47:26.524195 13911 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJN9WRF/build/release/bin/kudu with pid 17063
I20260924 13:47:26.528295 13911 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJN9WRF/build/release/bin/kudu with pid 17198
I20260924 13:47:26.533270 13911 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJN9WRF/build/release/bin/kudu with pid 17335
I20260924 13:47:26.537515 13911 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJN9WRF/build/release/bin/kudu with pid 16969
2026-09-24T13:47:26Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (2444 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790257629041645-13911-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 13:47:26 dist-test-slave-sjl2 krb5kdc[17480](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 13:47:26 dist-test-slave-sjl2 krb5kdc[17480](info): set up 2 sockets
Sep 24 13:47:26 dist-test-slave-sjl2 krb5kdc[17480](info): commencing operation
krb5kdc: starting...
W20260924 13:47:27.506035 13911 mini_kdc.cc:121] Time spent starting KDC: real 0.949s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790257629041645-13911-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790257629041645-13911-0/krb5kdc/test-user.keytab.
Sep 24 13:47:27 dist-test-slave-sjl2 krb5kdc[17480](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257647, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T13:47:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T13:47:27Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.149.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.149.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.149.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.254.keytab.
Entry for principal kudu/127.13.149.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.254.keytab.
I20260924 13:47:27.624253 13911 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJN9WRF/build/release/bin/kudu
/tmp/dist-test-taskJN9WRF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790257629041645-13911-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790257629041645-13911-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790257629041645-13911-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790257629041645-13911-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.149.254:32959
--webserver_interface=127.13.149.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.254.keytab
--principal=kudu/127.13.149.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.149.212:36709
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.149.254:32959
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790257629041645-13911-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790257629041645-13911-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790257629041645-13911-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 13:47:27.708107 17493 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 13:47:27.708345 17493 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 13:47:27.708370 17493 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 13:47:27.709646 17493 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 13:47:27.709699 17493 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 13:47:27.709713 17493 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 13:47:27.709725 17493 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 13:47:27.709736 17493 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 13:47:27.711346 17493 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.149.212:36709
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790257629041645-13911-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790257629041645-13911-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.149.254:32959
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.254.keytab
--principal=kudu/127.13.149.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.149.254:32959
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790257629041645-13911-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.149.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17493
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790257629041645-13911-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 12:29:37 UTC on ccycloud.quasar-iqzbxn.root.comops.site
build id 4670106
I20260924 13:47:27.711580 17493 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 13:47:27.711773 17493 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 13:47:27.716624 17499 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 13:47:27.716598 17498 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 13:47:27.716627 17501 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 13:47:27.716775 17493 server_base.cc:1034] running on GCE node
I20260924 13:47:27.717193 17493 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 13:47:27.718024 17493 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 13:47:27.719218 17493 hybrid_clock.cc:648] HybridClock initialized: now 1790257647719202 us; error 38 us; skew 500 ppm
Sep 24 13:47:27 dist-test-slave-sjl2 krb5kdc[17480](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257647, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.149.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 13:47:27.721994 17493 init.cc:377] Logged in from keytab as kudu/127.13.149.254@KRBTEST.COM (short username kudu)
I20260924 13:47:27.722491 17493 webserver.cc:460] Webserver started at http://127.13.149.254:45497/ using document root <none> and password file <none>
I20260924 13:47:27.722715 17493 fs_manager.cc:359] Metadata directory not provided
I20260924 13:47:27.722769 17493 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 13:47:27.722905 17493 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 13:47:27.724038 17493 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790257629041645-13911-0/minicluster-data/master-0/data/instance:
uuid: "f3ed641e5434434eaf625a40e7f1bace"
format_stamp: "Formatted at 2026-09-24 13:47:27 on dist-test-slave-sjl2"
server_key: "2273f18127e8c35d3a338d6234ec6451"
server_key_iv: "53bbc66e9db4c104ab2c12cd2c7656e7"
server_key_version: "encryptionkey@0"
I20260924 13:47:27.724406 17493 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790257629041645-13911-0/minicluster-data/master-0/wal/instance:
uuid: "f3ed641e5434434eaf625a40e7f1bace"
format_stamp: "Formatted at 2026-09-24 13:47:27 on dist-test-slave-sjl2"
server_key: "2273f18127e8c35d3a338d6234ec6451"
server_key_iv: "53bbc66e9db4c104ab2c12cd2c7656e7"
server_key_version: "encryptionkey@0"
I20260924 13:47:27.725894 17493 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260924 13:47:27.726655 17509 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 13:47:27.726814 17493 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 13:47:27.726902 17493 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790257629041645-13911-0/minicluster-data/master-0/data,/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790257629041645-13911-0/minicluster-data/master-0/wal
uuid: "f3ed641e5434434eaf625a40e7f1bace"
format_stamp: "Formatted at 2026-09-24 13:47:27 on dist-test-slave-sjl2"
server_key: "2273f18127e8c35d3a338d6234ec6451"
server_key_iv: "53bbc66e9db4c104ab2c12cd2c7656e7"
server_key_version: "encryptionkey@0"
I20260924 13:47:27.726966 17493 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790257629041645-13911-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790257629041645-13911-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790257629041645-13911-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 13:47:27.743145 17493 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 13:47:27.745417 17493 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 13:47:27.745570 17493 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 13:47:27.749562 17493 rpc_server.cc:307] RPC server started. Bound to: 127.13.149.254:32959
I20260924 13:47:27.749598 17561 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.149.254:32959 every 8 connection(s)
I20260924 13:47:27.750542 17493 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790257629041645-13911-0/minicluster-data/master-0/data/info.pb
I20260924 13:47:27.751616 17562 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 13:47:27.754107 17562 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f3ed641e5434434eaf625a40e7f1bace: Bootstrap starting.
I20260924 13:47:27.754756 17562 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f3ed641e5434434eaf625a40e7f1bace: Neither blocks nor log segments found. Creating new log.
I20260924 13:47:27.755045 17562 log.cc:824] T 00000000000000000000000000000000 P f3ed641e5434434eaf625a40e7f1bace: Log is configured to *not* fsync() on all Append() calls
I20260924 13:47:27.755919 17562 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f3ed641e5434434eaf625a40e7f1bace: No bootstrap required, opened a new log
I20260924 13:47:27.757642 17562 raft_consensus.cc:348] T 00000000000000000000000000000000 P f3ed641e5434434eaf625a40e7f1bace [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3ed641e5434434eaf625a40e7f1bace" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 32959 } }
I20260924 13:47:27.757822 17562 raft_consensus.cc:374] T 00000000000000000000000000000000 P f3ed641e5434434eaf625a40e7f1bace [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 13:47:27.757845 17562 raft_consensus.cc:729] T 00000000000000000000000000000000 P f3ed641e5434434eaf625a40e7f1bace [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3ed641e5434434eaf625a40e7f1bace, State: Initialized, Role: FOLLOWER
I20260924 13:47:27.757987 17562 consensus_queue.cc:260] T 00000000000000000000000000000000 P f3ed641e5434434eaf625a40e7f1bace [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3ed641e5434434eaf625a40e7f1bace" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 32959 } }
I20260924 13:47:27.758068 17562 raft_consensus.cc:388] T 00000000000000000000000000000000 P f3ed641e5434434eaf625a40e7f1bace [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 13:47:27.758098 17562 raft_consensus.cc:482] T 00000000000000000000000000000000 P f3ed641e5434434eaf625a40e7f1bace [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 13:47:27.758149 17562 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f3ed641e5434434eaf625a40e7f1bace [term 0 FOLLOWER]: Advancing to term 1
I20260924 13:47:27.759162 17562 raft_consensus.cc:504] T 00000000000000000000000000000000 P f3ed641e5434434eaf625a40e7f1bace [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3ed641e5434434eaf625a40e7f1bace" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 32959 } }
I20260924 13:47:27.759272 17562 leader_election.cc:302] T 00000000000000000000000000000000 P f3ed641e5434434eaf625a40e7f1bace [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f3ed641e5434434eaf625a40e7f1bace; no voters:
I20260924 13:47:27.759400 13911 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJN9WRF/build/release/bin/kudu as pid 17493
I20260924 13:47:27.759433 17562 leader_election.cc:288] T 00000000000000000000000000000000 P f3ed641e5434434eaf625a40e7f1bace [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 13:47:27.759474 13911 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790257629041645-13911-0/minicluster-data/master-0/wal/instance
I20260924 13:47:27.759500 17565 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f3ed641e5434434eaf625a40e7f1bace [term 1 FOLLOWER]: Leader election won for term 1
I20260924 13:47:27.759613 17565 raft_consensus.cc:686] T 00000000000000000000000000000000 P f3ed641e5434434eaf625a40e7f1bace [term 1 LEADER]: Becoming Leader. State: Replica: f3ed641e5434434eaf625a40e7f1bace, State: Running, Role: LEADER
I20260924 13:47:27.759640 13911 external_mini_cluster.cc:1428] Setting key 0859dbab0dc2e9771019a7481ec64e7b
I20260924 13:47:27.759693 17565 consensus_queue.cc:237] T 00000000000000000000000000000000 P f3ed641e5434434eaf625a40e7f1bace [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3ed641e5434434eaf625a40e7f1bace" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 32959 } }
I20260924 13:47:27.759732 17562 sys_catalog.cc:564] T 00000000000000000000000000000000 P f3ed641e5434434eaf625a40e7f1bace [sys.catalog]: configured and running, proceeding with master startup.
I20260924 13:47:27.760028 17565 sys_catalog.cc:455] T 00000000000000000000000000000000 P f3ed641e5434434eaf625a40e7f1bace [sys.catalog]: SysCatalogTable state changed. Reason: New leader f3ed641e5434434eaf625a40e7f1bace. Latest consensus state: current_term: 1 leader_uuid: "f3ed641e5434434eaf625a40e7f1bace" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3ed641e5434434eaf625a40e7f1bace" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 32959 } } }
I20260924 13:47:27.760104 17565 sys_catalog.cc:458] T 00000000000000000000000000000000 P f3ed641e5434434eaf625a40e7f1bace [sys.catalog]: This master's current role is: LEADER
I20260924 13:47:27.760046 17566 sys_catalog.cc:455] T 00000000000000000000000000000000 P f3ed641e5434434eaf625a40e7f1bace [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f3ed641e5434434eaf625a40e7f1bace" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3ed641e5434434eaf625a40e7f1bace" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 32959 } } }
I20260924 13:47:27.760275 17566 sys_catalog.cc:458] T 00000000000000000000000000000000 P f3ed641e5434434eaf625a40e7f1bace [sys.catalog]: This master's current role is: LEADER
I20260924 13:47:27.760330 17571 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 13:47:27.760890 17571 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 13:47:27.762519 17571 catalog_manager.cc:1347] Generated new cluster ID: e8311a0c491c4ccb9bf3794a4e9fe1b2
I20260924 13:47:27.762575 17571 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 24 13:47:27 dist-test-slave-sjl2 krb5kdc[17480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257647, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.149.254@KRBTEST.COM
I20260924 13:47:27.769629 17575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:27.760260 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52376 (local address 127.13.149.254:32959)
0924 13:47:27.760483 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0924 13:47:27.760493 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0924 13:47:27.760578 (+ 85us) server_negotiation.cc:407] Connection header received
0924 13:47:27.760783 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:47:27.760791 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:47:27.760851 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:47:27.761632 (+ 781us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:47:27.761895 (+ 263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:27.762983 (+ 1088us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:27.763751 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:27.764177 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:27.764269 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:47:27.766462 (+ 2193us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:47:27.766466 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:47:27.766467 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 13:47:27.766478 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:47:27.768052 (+ 1574us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:47:27.768386 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:47:27.768389 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:47:27.768389 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:47:27.768487 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:47:27.768663 (+ 176us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:47:27.768663 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:47:27.768664 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:47:27.768965 (+ 301us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:47:27.769130 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0924 13:47:27.769230 (+ 100us) server_negotiation.cc:299] Negotiation successful
0924 13:47:27.769381 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":171,"thread_start_us":90,"threads_started":1}
I20260924 13:47:27.786019 17571 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 13:47:27.786571 17571 catalog_manager.cc:1504] Loading token signing keys...
I20260924 13:47:27.791879 17571 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f3ed641e5434434eaf625a40e7f1bace: Generated new TSK 0
I20260924 13:47:27.792073 17571 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.13.149.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.149.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.149.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.193.keytab.
Entry for principal kudu/127.13.149.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.193.keytab.
I20260924 13:47:27.828845 13911 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJN9WRF/build/release/bin/kudu
/tmp/dist-test-taskJN9WRF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790257629041645-13911-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790257629041645-13911-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790257629041645-13911-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790257629041645-13911-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.149.193:0
--local_ip_for_outbound_sockets=127.13.149.193
--webserver_interface=127.13.149.193
--webserver_port=0
--tserver_master_addrs=127.13.149.254:32959
--keytab_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.193.keytab
--principal=kudu/127.13.149.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.149.212:36709
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790257629041645-13911-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790257629041645-13911-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790257629041645-13911-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 13:47:27.943142 17588 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 13:47:27.943416 17588 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 13:47:27.943449 17588 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 13:47:27.944723 17588 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 13:47:27.944774 17588 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 13:47:27.944803 17588 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.149.193
I20260924 13:47:27.946543 17588 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.149.212:36709
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790257629041645-13911-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790257629041645-13911-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.193.keytab
--principal=kudu/127.13.149.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.149.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790257629041645-13911-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.149.193
--webserver_port=0
--tserver_master_addrs=127.13.149.254:32959
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17588
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.149.193
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790257629041645-13911-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 12:29:37 UTC on ccycloud.quasar-iqzbxn.root.comops.site
build id 4670106
I20260924 13:47:27.946794 17588 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 13:47:27.946991 17588 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 13:47:27.952168 17596 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 13:47:27.952168 17594 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 13:47:27.952203 17593 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 13:47:27.952203 17588 server_base.cc:1034] running on GCE node
I20260924 13:47:27.952638 17588 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 13:47:27.952883 17588 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 13:47:27.954056 17588 hybrid_clock.cc:648] HybridClock initialized: now 1790257647954035 us; error 34 us; skew 500 ppm
Sep 24 13:47:27 dist-test-slave-sjl2 krb5kdc[17480](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257647, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.149.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 13:47:27.956676 17588 init.cc:377] Logged in from keytab as kudu/127.13.149.193@KRBTEST.COM (short username kudu)
I20260924 13:47:27.957224 17588 webserver.cc:460] Webserver started at http://127.13.149.193:38921/ using document root <none> and password file <none>
I20260924 13:47:27.957448 17588 fs_manager.cc:359] Metadata directory not provided
I20260924 13:47:27.957494 17588 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 13:47:27.957615 17588 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 13:47:27.958674 17588 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790257629041645-13911-0/minicluster-data/ts-0/data/instance:
uuid: "36fc5268a9bb4a34898bf42ef8f17da2"
format_stamp: "Formatted at 2026-09-24 13:47:27 on dist-test-slave-sjl2"
server_key: "d4d5462b99a2a7a608f76eb1aa0bf590"
server_key_iv: "8ef0a85e5bc687014b20f6d65cffa5bd"
server_key_version: "encryptionkey@0"
I20260924 13:47:27.959022 17588 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790257629041645-13911-0/minicluster-data/ts-0/wal/instance:
uuid: "36fc5268a9bb4a34898bf42ef8f17da2"
format_stamp: "Formatted at 2026-09-24 13:47:27 on dist-test-slave-sjl2"
server_key: "d4d5462b99a2a7a608f76eb1aa0bf590"
server_key_iv: "8ef0a85e5bc687014b20f6d65cffa5bd"
server_key_version: "encryptionkey@0"
I20260924 13:47:27.960377 17588 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260924 13:47:27.961210 17603 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 13:47:27.961354 17588 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260924 13:47:27.961438 17588 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790257629041645-13911-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790257629041645-13911-0/minicluster-data/ts-0/wal
uuid: "36fc5268a9bb4a34898bf42ef8f17da2"
format_stamp: "Formatted at 2026-09-24 13:47:27 on dist-test-slave-sjl2"
server_key: "d4d5462b99a2a7a608f76eb1aa0bf590"
server_key_iv: "8ef0a85e5bc687014b20f6d65cffa5bd"
server_key_version: "encryptionkey@0"
I20260924 13:47:27.961503 17588 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790257629041645-13911-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790257629041645-13911-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790257629041645-13911-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 13:47:27.970660 17588 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 13:47:27.972867 17588 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 13:47:27.973031 17588 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 13:47:27.973259 17588 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 13:47:27.973524 17588 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 13:47:27.973555 17588 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 13:47:27.973577 17588 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 13:47:27.973592 17588 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 13:47:27.979059 17588 rpc_server.cc:307] RPC server started. Bound to: 127.13.149.193:44075
I20260924 13:47:27.979156 17717 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.149.193:44075 every 8 connection(s)
I20260924 13:47:27.979439 17588 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790257629041645-13911-0/minicluster-data/ts-0/data/info.pb
I20260924 13:47:27.984840 13911 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJN9WRF/build/release/bin/kudu as pid 17588
I20260924 13:47:27.984977 13911 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790257629041645-13911-0/minicluster-data/ts-0/wal/instance
I20260924 13:47:27.985168 13911 external_mini_cluster.cc:1428] Setting key feff6c01b3888d8c22dd449b8021dfba
Sep 24 13:47:27 dist-test-slave-sjl2 krb5kdc[17480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257647, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.149.193@KRBTEST.COM for kudu/127.13.149.254@KRBTEST.COM
I20260924 13:47:27.989490 17575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:27.980288 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.149.193:58027 (local address 127.13.149.254:32959)
0924 13:47:27.980381 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0924 13:47:27.980383 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:47:27.980529 (+ 146us) server_negotiation.cc:407] Connection header received
0924 13:47:27.981510 (+ 981us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:47:27.981512 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:47:27.981526 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:47:27.981578 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:47:27.982238 (+ 660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:27.982757 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:27.983763 (+ 1006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:27.983969 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:27.984025 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:47:27.986488 (+ 2463us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:47:27.986489 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:47:27.986490 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 13:47:27.986497 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:47:27.988103 (+ 1606us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:47:27.988483 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:47:27.988484 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:47:27.988485 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:47:27.988543 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:47:27.988817 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:47:27.988818 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:47:27.988818 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:47:27.988973 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:47:27.989033 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0924 13:47:27.989300 (+ 267us) server_negotiation.cc:299] Negotiation successful
0924 13:47:27.989393 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260924 13:47:27.989671 17720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:27.980221 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:32959 (local address 127.13.149.193:58027)
0924 13:47:27.980402 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0924 13:47:27.980446 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0924 13:47:27.981397 (+ 951us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:47:27.981648 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:47:27.981652 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:47:27.981940 (+ 288us) client_negotiation.cc:189] Negotiated authn=SASL
0924 13:47:27.982136 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:47:27.982140 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:47:27.982848 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:47:27.982849 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:47:27.983671 (+ 822us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:47:27.983673 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:47:27.984042 (+ 369us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:47:27.984043 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:47:27.984177 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:47:27.984768 (+ 591us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 13:47:27.984776 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0924 13:47:27.986384 (+ 1608us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 13:47:27.988171 (+ 1787us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 13:47:27.988181 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 13:47:27.988184 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0924 13:47:27.988415 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 13:47:27.988597 (+ 182us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 13:47:27.988598 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 13:47:27.988598 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 13:47:27.988755 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 13:47:27.989022 (+ 267us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 13:47:27.989029 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 13:47:27.989175 (+ 146us) client_negotiation.cc:769] Sending connection context
0924 13:47:27.989270 (+ 95us) client_negotiation.cc:240] Negotiation successful
0924 13:47:27.989445 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":82,"threads_started":1}
I20260924 13:47:27.990144 17718 heartbeater.cc:347] Connected to a master server at 127.13.149.254:32959
I20260924 13:47:27.990239 17718 heartbeater.cc:464] Registering TS with master...
I20260924 13:47:27.990417 17718 heartbeater.cc:511] Master 127.13.149.254:32959 requested a full tablet report, sending...
I20260924 13:47:27.990990 17526 ts_manager.cc:194] Registered new tserver with Master: 36fc5268a9bb4a34898bf42ef8f17da2 (127.13.149.193:44075)
I20260924 13:47:27.991624 17526 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.149.193@KRBTEST.COM'} at 127.13.149.193:58027
WARNING: no policy specified for kudu/127.13.149.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.149.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.149.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.194.keytab.
Entry for principal kudu/127.13.149.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.194.keytab.
I20260924 13:47:28.012385 13911 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJN9WRF/build/release/bin/kudu
/tmp/dist-test-taskJN9WRF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790257629041645-13911-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790257629041645-13911-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790257629041645-13911-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790257629041645-13911-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.149.194:0
--local_ip_for_outbound_sockets=127.13.149.194
--webserver_interface=127.13.149.194
--webserver_port=0
--tserver_master_addrs=127.13.149.254:32959
--keytab_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.194.keytab
--principal=kudu/127.13.149.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.149.212:36709
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790257629041645-13911-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790257629041645-13911-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790257629041645-13911-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 13:47:28.092885 17723 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 13:47:28.093152 17723 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 13:47:28.093178 17723 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 13:47:28.094389 17723 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 13:47:28.094437 17723 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 13:47:28.094466 17723 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.149.194
I20260924 13:47:28.096014 17723 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.149.212:36709
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790257629041645-13911-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790257629041645-13911-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.194.keytab
--principal=kudu/127.13.149.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.149.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790257629041645-13911-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.149.194
--webserver_port=0
--tserver_master_addrs=127.13.149.254:32959
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17723
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.149.194
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790257629041645-13911-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 12:29:37 UTC on ccycloud.quasar-iqzbxn.root.comops.site
build id 4670106
I20260924 13:47:28.096251 17723 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 13:47:28.096480 17723 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 13:47:28.101480 17729 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 13:47:28.101552 17723 server_base.cc:1034] running on GCE node
W20260924 13:47:28.101478 17731 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 13:47:28.101485 17728 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 13:47:28.101905 17723 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 13:47:28.102121 17723 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 13:47:28.103276 17723 hybrid_clock.cc:648] HybridClock initialized: now 1790257648103268 us; error 34 us; skew 500 ppm
Sep 24 13:47:28 dist-test-slave-sjl2 krb5kdc[17480](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257648, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.149.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 13:47:28.105865 17723 init.cc:377] Logged in from keytab as kudu/127.13.149.194@KRBTEST.COM (short username kudu)
I20260924 13:47:28.106335 17723 webserver.cc:460] Webserver started at http://127.13.149.194:37951/ using document root <none> and password file <none>
I20260924 13:47:28.106542 17723 fs_manager.cc:359] Metadata directory not provided
I20260924 13:47:28.106586 17723 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 13:47:28.106714 17723 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 13:47:28.107776 17723 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790257629041645-13911-0/minicluster-data/ts-1/data/instance:
uuid: "0d9f2c788fa149ea9e3a09476c7cdc66"
format_stamp: "Formatted at 2026-09-24 13:47:28 on dist-test-slave-sjl2"
server_key: "f0616c10671b3a4990bc027f0807c742"
server_key_iv: "54449ad0b6e83feb508bcd50e6e87306"
server_key_version: "encryptionkey@0"
I20260924 13:47:28.108116 17723 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790257629041645-13911-0/minicluster-data/ts-1/wal/instance:
uuid: "0d9f2c788fa149ea9e3a09476c7cdc66"
format_stamp: "Formatted at 2026-09-24 13:47:28 on dist-test-slave-sjl2"
server_key: "f0616c10671b3a4990bc027f0807c742"
server_key_iv: "54449ad0b6e83feb508bcd50e6e87306"
server_key_version: "encryptionkey@0"
I20260924 13:47:28.109557 17723 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260924 13:47:28.110317 17738 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 13:47:28.110455 17723 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 13:47:28.110538 17723 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790257629041645-13911-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790257629041645-13911-0/minicluster-data/ts-1/wal
uuid: "0d9f2c788fa149ea9e3a09476c7cdc66"
format_stamp: "Formatted at 2026-09-24 13:47:28 on dist-test-slave-sjl2"
server_key: "f0616c10671b3a4990bc027f0807c742"
server_key_iv: "54449ad0b6e83feb508bcd50e6e87306"
server_key_version: "encryptionkey@0"
I20260924 13:47:28.110602 17723 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790257629041645-13911-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790257629041645-13911-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790257629041645-13911-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 13:47:28.132407 17723 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 13:47:28.134570 17723 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 13:47:28.134753 17723 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 13:47:28.134989 17723 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 13:47:28.135288 17723 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 13:47:28.135321 17723 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 13:47:28.135351 17723 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 13:47:28.135368 17723 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 13:47:28.141042 17723 rpc_server.cc:307] RPC server started. Bound to: 127.13.149.194:46429
I20260924 13:47:28.141091 17852 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.149.194:46429 every 8 connection(s)
I20260924 13:47:28.141438 17723 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790257629041645-13911-0/minicluster-data/ts-1/data/info.pb
I20260924 13:47:28.147452 13911 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJN9WRF/build/release/bin/kudu as pid 17723
I20260924 13:47:28.147552 13911 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790257629041645-13911-0/minicluster-data/ts-1/wal/instance
Sep 24 13:47:28 dist-test-slave-sjl2 krb5kdc[17480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257648, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.149.194@KRBTEST.COM for kudu/127.13.149.254@KRBTEST.COM
I20260924 13:47:28.147728 13911 external_mini_cluster.cc:1428] Setting key da4b463a4d311063ba962855222ded68
I20260924 13:47:28.151026 17575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:28.142191 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.149.194:56743 (local address 127.13.149.254:32959)
0924 13:47:28.142257 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0924 13:47:28.142260 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 13:47:28.142454 (+ 194us) server_negotiation.cc:407] Connection header received
0924 13:47:28.143432 (+ 978us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:47:28.143433 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:47:28.143445 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:47:28.143507 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:47:28.144145 (+ 638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:28.144648 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:28.145726 (+ 1078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:28.145933 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:28.146000 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:47:28.148208 (+ 2208us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:47:28.148209 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:47:28.148210 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 13:47:28.148217 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:47:28.149734 (+ 1517us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:47:28.150135 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:47:28.150136 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:47:28.150137 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:47:28.150200 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:47:28.150401 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:47:28.150401 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:47:28.150402 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:47:28.150535 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:47:28.150599 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0924 13:47:28.150836 (+ 237us) server_negotiation.cc:299] Negotiation successful
0924 13:47:28.150932 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260924 13:47:28.151124 17855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:28.142166 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:32959 (local address 127.13.149.194:56743)
0924 13:47:28.142332 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0924 13:47:28.142369 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0924 13:47:28.143278 (+ 909us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:47:28.143579 (+ 301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:47:28.143582 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:47:28.143871 (+ 289us) client_negotiation.cc:189] Negotiated authn=SASL
0924 13:47:28.144068 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:47:28.144073 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:47:28.144715 (+ 642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:47:28.144716 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:47:28.145607 (+ 891us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:47:28.145610 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:47:28.146011 (+ 401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:47:28.146012 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:47:28.146121 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:47:28.146623 (+ 502us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 13:47:28.146631 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0924 13:47:28.148117 (+ 1486us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 13:47:28.149829 (+ 1712us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 13:47:28.149838 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 13:47:28.149840 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 13:47:28.150065 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 13:47:28.150255 (+ 190us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 13:47:28.150255 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 13:47:28.150255 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 13:47:28.150350 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 13:47:28.150590 (+ 240us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 13:47:28.150593 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 13:47:28.150713 (+ 120us) client_negotiation.cc:769] Sending connection context
0924 13:47:28.150816 (+ 103us) client_negotiation.cc:240] Negotiation successful
0924 13:47:28.150944 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":84,"threads_started":1}
I20260924 13:47:28.151579 17853 heartbeater.cc:347] Connected to a master server at 127.13.149.254:32959
I20260924 13:47:28.151687 17853 heartbeater.cc:464] Registering TS with master...
I20260924 13:47:28.151880 17853 heartbeater.cc:511] Master 127.13.149.254:32959 requested a full tablet report, sending...
I20260924 13:47:28.152284 17526 ts_manager.cc:194] Registered new tserver with Master: 0d9f2c788fa149ea9e3a09476c7cdc66 (127.13.149.194:46429)
I20260924 13:47:28.152806 17526 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.149.194@KRBTEST.COM'} at 127.13.149.194:56743
WARNING: no policy specified for kudu/127.13.149.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.149.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.149.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.195.keytab.
Entry for principal kudu/127.13.149.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.195.keytab.
I20260924 13:47:28.175819 13911 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJN9WRF/build/release/bin/kudu
/tmp/dist-test-taskJN9WRF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790257629041645-13911-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790257629041645-13911-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790257629041645-13911-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790257629041645-13911-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.149.195:0
--local_ip_for_outbound_sockets=127.13.149.195
--webserver_interface=127.13.149.195
--webserver_port=0
--tserver_master_addrs=127.13.149.254:32959
--keytab_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.195.keytab
--principal=kudu/127.13.149.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.149.212:36709
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790257629041645-13911-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790257629041645-13911-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790257629041645-13911-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 13:47:28.257422 17858 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 13:47:28.257663 17858 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 13:47:28.257694 17858 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 13:47:28.258955 17858 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 13:47:28.259001 17858 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 13:47:28.259032 17858 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.149.195
I20260924 13:47:28.261184 17858 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.149.212:36709
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790257629041645-13911-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790257629041645-13911-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.195.keytab
--principal=kudu/127.13.149.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.149.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790257629041645-13911-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.149.195
--webserver_port=0
--tserver_master_addrs=127.13.149.254:32959
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17858
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.149.195
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790257629041645-13911-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 12:29:37 UTC on ccycloud.quasar-iqzbxn.root.comops.site
build id 4670106
I20260924 13:47:28.261497 17858 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 13:47:28.261711 17858 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 13:47:28.266985 17863 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 13:47:28.267082 17866 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 13:47:28.266985 17864 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 13:47:28.267231 17858 server_base.cc:1034] running on GCE node
I20260924 13:47:28.267495 17858 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 13:47:28.267814 17858 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 13:47:28.269011 17858 hybrid_clock.cc:648] HybridClock initialized: now 1790257648268990 us; error 30 us; skew 500 ppm
Sep 24 13:47:28 dist-test-slave-sjl2 krb5kdc[17480](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257648, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.149.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 13:47:28.271790 17858 init.cc:377] Logged in from keytab as kudu/127.13.149.195@KRBTEST.COM (short username kudu)
I20260924 13:47:28.272290 17858 webserver.cc:460] Webserver started at http://127.13.149.195:41341/ using document root <none> and password file <none>
I20260924 13:47:28.272507 17858 fs_manager.cc:359] Metadata directory not provided
I20260924 13:47:28.272558 17858 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 13:47:28.272698 17858 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 13:47:28.273855 17858 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790257629041645-13911-0/minicluster-data/ts-2/data/instance:
uuid: "e49fcd5c5284449d9126e9dfdba72562"
format_stamp: "Formatted at 2026-09-24 13:47:28 on dist-test-slave-sjl2"
server_key: "25b2a108f873ab3534b5ad992383196e"
server_key_iv: "0603c328bd2a16f9850a39d44d43bbf7"
server_key_version: "encryptionkey@0"
I20260924 13:47:28.274207 17858 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790257629041645-13911-0/minicluster-data/ts-2/wal/instance:
uuid: "e49fcd5c5284449d9126e9dfdba72562"
format_stamp: "Formatted at 2026-09-24 13:47:28 on dist-test-slave-sjl2"
server_key: "25b2a108f873ab3534b5ad992383196e"
server_key_iv: "0603c328bd2a16f9850a39d44d43bbf7"
server_key_version: "encryptionkey@0"
I20260924 13:47:28.275622 17858 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260924 13:47:28.276520 17873 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 13:47:28.276700 17858 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260924 13:47:28.276803 17858 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790257629041645-13911-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790257629041645-13911-0/minicluster-data/ts-2/wal
uuid: "e49fcd5c5284449d9126e9dfdba72562"
format_stamp: "Formatted at 2026-09-24 13:47:28 on dist-test-slave-sjl2"
server_key: "25b2a108f873ab3534b5ad992383196e"
server_key_iv: "0603c328bd2a16f9850a39d44d43bbf7"
server_key_version: "encryptionkey@0"
I20260924 13:47:28.276877 17858 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790257629041645-13911-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790257629041645-13911-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790257629041645-13911-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 13:47:28.307806 17858 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 13:47:28.310009 17858 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 13:47:28.310158 17858 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 13:47:28.310389 17858 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 13:47:28.310726 17858 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 13:47:28.310761 17858 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 13:47:28.310792 17858 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 13:47:28.310809 17858 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 13:47:28.316385 17858 rpc_server.cc:307] RPC server started. Bound to: 127.13.149.195:34997
I20260924 13:47:28.316447 17988 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.149.195:34997 every 8 connection(s)
I20260924 13:47:28.316763 17858 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790257629041645-13911-0/minicluster-data/ts-2/data/info.pb
I20260924 13:47:28.321106 13911 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJN9WRF/build/release/bin/kudu as pid 17858
I20260924 13:47:28.321204 13911 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790257629041645-13911-0/minicluster-data/ts-2/wal/instance
I20260924 13:47:28.321374 13911 external_mini_cluster.cc:1428] Setting key 0f988b22d259811f1e9f87b309a93344
Sep 24 13:47:28 dist-test-slave-sjl2 krb5kdc[17480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257648, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.149.195@KRBTEST.COM for kudu/127.13.149.254@KRBTEST.COM
I20260924 13:47:28.326133 17575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:28.317494 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.149.195:33247 (local address 127.13.149.254:32959)
0924 13:47:28.317577 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0924 13:47:28.317579 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:47:28.317746 (+ 167us) server_negotiation.cc:407] Connection header received
0924 13:47:28.318663 (+ 917us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:47:28.318664 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:47:28.318677 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:47:28.318736 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:47:28.319342 (+ 606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:28.319855 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:28.320842 (+ 987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:28.321074 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:28.321173 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:47:28.323440 (+ 2267us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:47:28.323441 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:47:28.323441 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 13:47:28.323448 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:47:28.324564 (+ 1116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:47:28.324990 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:47:28.324991 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:47:28.324992 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:47:28.325047 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:47:28.325285 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:47:28.325286 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:47:28.325286 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:47:28.325418 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:47:28.325480 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0924 13:47:28.325897 (+ 417us) server_negotiation.cc:299] Negotiation successful
0924 13:47:28.326001 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260924 13:47:28.326363 17991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:28.317471 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:32959 (local address 127.13.149.195:33247)
0924 13:47:28.317625 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0924 13:47:28.317662 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0924 13:47:28.318541 (+ 879us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:47:28.318809 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:47:28.318813 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:47:28.319078 (+ 265us) client_negotiation.cc:189] Negotiated authn=SASL
0924 13:47:28.319275 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:47:28.319279 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:47:28.319921 (+ 642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:47:28.319922 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:47:28.320734 (+ 812us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:47:28.320736 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:47:28.321192 (+ 456us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:47:28.321193 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:47:28.321335 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:47:28.321927 (+ 592us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 13:47:28.321938 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0924 13:47:28.323358 (+ 1420us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 13:47:28.324635 (+ 1277us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 13:47:28.324641 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 13:47:28.324643 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 13:47:28.324892 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 13:47:28.325096 (+ 204us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 13:47:28.325097 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 13:47:28.325097 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 13:47:28.325199 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 13:47:28.325503 (+ 304us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 13:47:28.325510 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 13:47:28.325693 (+ 183us) client_negotiation.cc:769] Sending connection context
0924 13:47:28.325895 (+ 202us) client_negotiation.cc:240] Negotiation successful
0924 13:47:28.326101 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":76,"threads_started":1}
I20260924 13:47:28.326898 17989 heartbeater.cc:347] Connected to a master server at 127.13.149.254:32959
I20260924 13:47:28.327004 17989 heartbeater.cc:464] Registering TS with master...
I20260924 13:47:28.327172 17989 heartbeater.cc:511] Master 127.13.149.254:32959 requested a full tablet report, sending...
I20260924 13:47:28.327523 17526 ts_manager.cc:194] Registered new tserver with Master: e49fcd5c5284449d9126e9dfdba72562 (127.13.149.195:34997)
I20260924 13:47:28.327917 17526 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.149.195@KRBTEST.COM'} at 127.13.149.195:33247
I20260924 13:47:28.333242 13911 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 24 13:47:28 dist-test-slave-sjl2 krb5kdc[17480](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257648, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 24 13:47:28 dist-test-slave-sjl2 krb5kdc[17480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257648, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.149.193@KRBTEST.COM
I20260924 13:47:28.355475 17996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:28.347804 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52468 (local address 127.13.149.193:44075)
0924 13:47:28.347980 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0924 13:47:28.347985 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 13:47:28.347996 (+ 11us) server_negotiation.cc:407] Connection header received
0924 13:47:28.348022 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:47:28.348024 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:47:28.348038 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:47:28.348108 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:47:28.348645 (+ 537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:28.349447 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:28.350232 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:28.350551 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:28.350601 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:47:28.352857 (+ 2256us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:47:28.352870 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:47:28.352870 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 13:47:28.352878 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:47:28.354338 (+ 1460us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:47:28.354659 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:47:28.354662 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:47:28.354662 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:47:28.354730 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:47:28.354870 (+ 140us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:47:28.354870 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:47:28.354870 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:47:28.355004 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:47:28.355159 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0924 13:47:28.355283 (+ 124us) server_negotiation.cc:299] Negotiation successful
0924 13:47:28.355362 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":139,"thread_start_us":94,"threads_started":1}
Sep 24 13:47:28 dist-test-slave-sjl2 krb5kdc[17480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257648, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.149.254@KRBTEST.COM
I20260924 13:47:28.363543 17575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:28.356956 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52388 (local address 127.13.149.254:32959)
0924 13:47:28.357009 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0924 13:47:28.357010 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0924 13:47:28.357020 (+ 10us) server_negotiation.cc:407] Connection header received
0924 13:47:28.357057 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:47:28.357057 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:47:28.357067 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:47:28.357103 (+ 36us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:47:28.357551 (+ 448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:28.358070 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:28.358741 (+ 671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:28.358936 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:28.359005 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:47:28.361226 (+ 2221us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:47:28.361228 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:47:28.361229 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 13:47:28.361237 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:47:28.362497 (+ 1260us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:47:28.362888 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:47:28.362889 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:47:28.362889 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:47:28.362937 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:47:28.363096 (+ 159us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:47:28.363096 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:47:28.363096 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:47:28.363242 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:47:28.363324 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0924 13:47:28.363416 (+ 92us) server_negotiation.cc:299] Negotiation successful
0924 13:47:28.363474 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
W20260924 13:47:28.363950 17526 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52388
I20260924 13:47:28.364339 13911 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJN9WRF/build/release/bin/kudu with pid 17588
I20260924 13:47:28.368897 13911 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJN9WRF/build/release/bin/kudu with pid 17723
I20260924 13:47:28.373435 13911 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJN9WRF/build/release/bin/kudu with pid 17858
I20260924 13:47:28.377251 13911 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJN9WRF/build/release/bin/kudu with pid 17493
2026-09-24T13:47:28Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (1839 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790257629041645-13911-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 13:47:28 dist-test-slave-sjl2 krb5kdc[18001](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 13:47:28 dist-test-slave-sjl2 krb5kdc[18001](info): set up 2 sockets
Sep 24 13:47:28 dist-test-slave-sjl2 krb5kdc[18001](info): commencing operation
krb5kdc: starting...
W20260924 13:47:29.341898 13911 mini_kdc.cc:121] Time spent starting KDC: real 0.945s user 0.000s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790257629041645-13911-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790257629041645-13911-0/krb5kdc/test-user.keytab.
Sep 24 13:47:29 dist-test-slave-sjl2 krb5kdc[18001](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257649, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T13:47:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T13:47:29Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.149.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.149.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.149.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.254.keytab.
Entry for principal kudu/127.13.149.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.254.keytab.
I20260924 13:47:29.458968 13911 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJN9WRF/build/release/bin/kudu
/tmp/dist-test-taskJN9WRF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790257629041645-13911-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790257629041645-13911-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790257629041645-13911-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790257629041645-13911-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.149.254:40811
--webserver_interface=127.13.149.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.254.keytab
--principal=kudu/127.13.149.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.149.212:39509
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.149.254:40811
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790257629041645-13911-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790257629041645-13911-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790257629041645-13911-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 13:47:29.544369 18014 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 13:47:29.544611 18014 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 13:47:29.544636 18014 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 13:47:29.545930 18014 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 13:47:29.545975 18014 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 13:47:29.545989 18014 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 13:47:29.546000 18014 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 13:47:29.546011 18014 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 13:47:29.547613 18014 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.149.212:39509
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790257629041645-13911-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790257629041645-13911-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.149.254:40811
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.254.keytab
--principal=kudu/127.13.149.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.149.254:40811
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790257629041645-13911-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.149.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18014
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790257629041645-13911-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 12:29:37 UTC on ccycloud.quasar-iqzbxn.root.comops.site
build id 4670106
I20260924 13:47:29.547832 18014 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 13:47:29.548022 18014 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 13:47:29.552831 18020 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 13:47:29.552887 18022 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 13:47:29.552840 18019 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 13:47:29.553176 18014 server_base.cc:1034] running on GCE node
I20260924 13:47:29.553486 18014 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 13:47:29.553808 18014 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 13:47:29.554975 18014 hybrid_clock.cc:648] HybridClock initialized: now 1790257649554954 us; error 35 us; skew 500 ppm
Sep 24 13:47:29 dist-test-slave-sjl2 krb5kdc[18001](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257649, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.149.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 13:47:29.557559 18014 init.cc:377] Logged in from keytab as kudu/127.13.149.254@KRBTEST.COM (short username kudu)
I20260924 13:47:29.557895 18014 fs_manager.cc:359] Metadata directory not provided
I20260924 13:47:29.557945 18014 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 13:47:29.558076 18014 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 13:47:29.559176 18014 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790257629041645-13911-0/minicluster-data/master-0/data/instance:
uuid: "3022da7e5ed242b29d69bd84984ac2ab"
format_stamp: "Formatted at 2026-09-24 13:47:29 on dist-test-slave-sjl2"
server_key: "de98aa277b2998bbb3b005b54742ad63"
server_key_iv: "ba05d6de6e9e1b856e536d40633e6231"
server_key_version: "encryptionkey@0"
I20260924 13:47:29.559517 18014 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790257629041645-13911-0/minicluster-data/master-0/wal/instance:
uuid: "3022da7e5ed242b29d69bd84984ac2ab"
format_stamp: "Formatted at 2026-09-24 13:47:29 on dist-test-slave-sjl2"
server_key: "de98aa277b2998bbb3b005b54742ad63"
server_key_iv: "ba05d6de6e9e1b856e536d40633e6231"
server_key_version: "encryptionkey@0"
I20260924 13:47:29.560848 18014 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260924 13:47:29.561681 18028 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 13:47:29.561877 18014 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260924 13:47:29.561968 18014 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790257629041645-13911-0/minicluster-data/master-0/data,/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790257629041645-13911-0/minicluster-data/master-0/wal
uuid: "3022da7e5ed242b29d69bd84984ac2ab"
format_stamp: "Formatted at 2026-09-24 13:47:29 on dist-test-slave-sjl2"
server_key: "de98aa277b2998bbb3b005b54742ad63"
server_key_iv: "ba05d6de6e9e1b856e536d40633e6231"
server_key_version: "encryptionkey@0"
I20260924 13:47:29.562036 18014 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790257629041645-13911-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790257629041645-13911-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790257629041645-13911-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 13:47:29.570104 18014 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 13:47:29.572845 18014 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 13:47:29.573030 18014 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 13:47:29.577030 18014 rpc_server.cc:307] RPC server started. Bound to: 127.13.149.254:40811
I20260924 13:47:29.577057 18081 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.149.254:40811 every 8 connection(s)
I20260924 13:47:29.577358 18014 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790257629041645-13911-0/minicluster-data/master-0/data/info.pb
I20260924 13:47:29.578269 18082 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 13:47:29.580691 18082 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3022da7e5ed242b29d69bd84984ac2ab: Bootstrap starting.
I20260924 13:47:29.581346 18082 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3022da7e5ed242b29d69bd84984ac2ab: Neither blocks nor log segments found. Creating new log.
I20260924 13:47:29.581616 18082 log.cc:824] T 00000000000000000000000000000000 P 3022da7e5ed242b29d69bd84984ac2ab: Log is configured to *not* fsync() on all Append() calls
I20260924 13:47:29.582273 18082 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3022da7e5ed242b29d69bd84984ac2ab: No bootstrap required, opened a new log
I20260924 13:47:29.583954 13911 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJN9WRF/build/release/bin/kudu as pid 18014
I20260924 13:47:29.584085 13911 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790257629041645-13911-0/minicluster-data/master-0/wal/instance
I20260924 13:47:29.583997 18082 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3022da7e5ed242b29d69bd84984ac2ab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3022da7e5ed242b29d69bd84984ac2ab" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 40811 } }
I20260924 13:47:29.584182 18082 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3022da7e5ed242b29d69bd84984ac2ab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 13:47:29.584223 18082 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3022da7e5ed242b29d69bd84984ac2ab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3022da7e5ed242b29d69bd84984ac2ab, State: Initialized, Role: FOLLOWER
I20260924 13:47:29.584275 13911 external_mini_cluster.cc:1428] Setting key f4b2800d5103b291999a2f9f6d688749
I20260924 13:47:29.584373 18082 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3022da7e5ed242b29d69bd84984ac2ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3022da7e5ed242b29d69bd84984ac2ab" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 40811 } }
I20260924 13:47:29.584457 18082 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3022da7e5ed242b29d69bd84984ac2ab [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 13:47:29.584499 18082 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3022da7e5ed242b29d69bd84984ac2ab [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 13:47:29.584547 18082 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3022da7e5ed242b29d69bd84984ac2ab [term 0 FOLLOWER]: Advancing to term 1
I20260924 13:47:29.585528 18082 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3022da7e5ed242b29d69bd84984ac2ab [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3022da7e5ed242b29d69bd84984ac2ab" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 40811 } }
I20260924 13:47:29.585633 18082 leader_election.cc:302] T 00000000000000000000000000000000 P 3022da7e5ed242b29d69bd84984ac2ab [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3022da7e5ed242b29d69bd84984ac2ab; no voters:
I20260924 13:47:29.585798 18082 leader_election.cc:288] T 00000000000000000000000000000000 P 3022da7e5ed242b29d69bd84984ac2ab [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 13:47:29.585835 18087 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3022da7e5ed242b29d69bd84984ac2ab [term 1 FOLLOWER]: Leader election won for term 1
I20260924 13:47:29.585932 18087 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3022da7e5ed242b29d69bd84984ac2ab [term 1 LEADER]: Becoming Leader. State: Replica: 3022da7e5ed242b29d69bd84984ac2ab, State: Running, Role: LEADER
I20260924 13:47:29.586010 18087 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3022da7e5ed242b29d69bd84984ac2ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3022da7e5ed242b29d69bd84984ac2ab" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 40811 } }
I20260924 13:47:29.586069 18082 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3022da7e5ed242b29d69bd84984ac2ab [sys.catalog]: configured and running, proceeding with master startup.
I20260924 13:47:29.586390 18087 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3022da7e5ed242b29d69bd84984ac2ab [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3022da7e5ed242b29d69bd84984ac2ab" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3022da7e5ed242b29d69bd84984ac2ab" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 40811 } } }
I20260924 13:47:29.586469 18087 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3022da7e5ed242b29d69bd84984ac2ab [sys.catalog]: This master's current role is: LEADER
I20260924 13:47:29.586580 18087 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3022da7e5ed242b29d69bd84984ac2ab [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3022da7e5ed242b29d69bd84984ac2ab. Latest consensus state: current_term: 1 leader_uuid: "3022da7e5ed242b29d69bd84984ac2ab" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3022da7e5ed242b29d69bd84984ac2ab" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 40811 } } }
I20260924 13:47:29.586632 18087 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3022da7e5ed242b29d69bd84984ac2ab [sys.catalog]: This master's current role is: LEADER
I20260924 13:47:29.587203 18093 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 13:47:29.587707 18093 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 13:47:29.589178 18093 catalog_manager.cc:1347] Generated new cluster ID: acf22d793780470a9a92307f20f8376c
I20260924 13:47:29.589231 18093 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 24 13:47:29 dist-test-slave-sjl2 krb5kdc[18001](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257649, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.149.254@KRBTEST.COM
I20260924 13:47:29.599443 18093 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 13:47:29.600139 18093 catalog_manager.cc:1504] Loading token signing keys...
I20260924 13:47:29.603116 18086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:29.584779 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48976 (local address 127.13.149.254:40811)
0924 13:47:29.584997 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0924 13:47:29.585007 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0924 13:47:29.585041 (+ 34us) server_negotiation.cc:407] Connection header received
0924 13:47:29.585148 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:47:29.585154 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:47:29.585204 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:47:29.585332 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:47:29.586047 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:29.587435 (+ 1388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:29.588662 (+ 1227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:29.596145 (+ 7483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:29.596310 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:47:29.599329 (+ 3019us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:47:29.599336 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:47:29.599337 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 13:47:29.599353 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:47:29.601349 (+ 1996us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:47:29.601843 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:47:29.601847 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:47:29.601847 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:47:29.601976 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:47:29.602172 (+ 196us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:47:29.602172 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:47:29.602173 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:47:29.602456 (+ 283us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:47:29.602601 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0924 13:47:29.602716 (+ 115us) server_negotiation.cc:299] Negotiation successful
0924 13:47:29.602877 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":163,"thread_start_us":85,"threads_started":1}
I20260924 13:47:29.604516 18093 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3022da7e5ed242b29d69bd84984ac2ab: Generated new TSK 0
I20260924 13:47:29.604722 18093 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.13.149.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.149.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.149.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.193.keytab.
Entry for principal kudu/127.13.149.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.193.keytab.
I20260924 13:47:29.636003 13911 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJN9WRF/build/release/bin/kudu
/tmp/dist-test-taskJN9WRF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790257629041645-13911-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790257629041645-13911-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790257629041645-13911-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790257629041645-13911-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.149.193:0
--local_ip_for_outbound_sockets=127.13.149.193
--webserver_interface=127.13.149.193
--webserver_port=0
--tserver_master_addrs=127.13.149.254:40811
--keytab_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.193.keytab
--principal=kudu/127.13.149.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.149.212:39509
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790257629041645-13911-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790257629041645-13911-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790257629041645-13911-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 13:47:29.720281 18108 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 13:47:29.720508 18108 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 13:47:29.720546 18108 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 13:47:29.721859 18108 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 13:47:29.721920 18108 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 13:47:29.721961 18108 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.149.193
I20260924 13:47:29.723703 18108 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.149.212:39509
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790257629041645-13911-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790257629041645-13911-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.193.keytab
--principal=kudu/127.13.149.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.149.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790257629041645-13911-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.149.193
--webserver_port=0
--tserver_master_addrs=127.13.149.254:40811
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18108
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.149.193
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790257629041645-13911-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 12:29:37 UTC on ccycloud.quasar-iqzbxn.root.comops.site
build id 4670106
I20260924 13:47:29.723979 18108 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 13:47:29.724203 18108 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 13:47:29.729264 18113 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 13:47:29.729270 18114 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 13:47:29.729264 18116 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 13:47:29.729553 18108 server_base.cc:1034] running on GCE node
I20260924 13:47:29.729839 18108 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 13:47:29.730162 18108 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 13:47:29.731356 18108 hybrid_clock.cc:648] HybridClock initialized: now 1790257649731337 us; error 36 us; skew 500 ppm
Sep 24 13:47:29 dist-test-slave-sjl2 krb5kdc[18001](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257649, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.149.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 13:47:29.734288 18108 init.cc:377] Logged in from keytab as kudu/127.13.149.193@KRBTEST.COM (short username kudu)
I20260924 13:47:29.734681 18108 fs_manager.cc:359] Metadata directory not provided
I20260924 13:47:29.734740 18108 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 13:47:29.734870 18108 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 13:47:29.736044 18108 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790257629041645-13911-0/minicluster-data/ts-0/data/instance:
uuid: "aff50161196a4d46ad09d8617159bf3b"
format_stamp: "Formatted at 2026-09-24 13:47:29 on dist-test-slave-sjl2"
server_key: "b6649d2e053061ff7266943a954e6e73"
server_key_iv: "794eb12aa77233b33c2e4307b8e36cae"
server_key_version: "encryptionkey@0"
I20260924 13:47:29.736397 18108 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790257629041645-13911-0/minicluster-data/ts-0/wal/instance:
uuid: "aff50161196a4d46ad09d8617159bf3b"
format_stamp: "Formatted at 2026-09-24 13:47:29 on dist-test-slave-sjl2"
server_key: "b6649d2e053061ff7266943a954e6e73"
server_key_iv: "794eb12aa77233b33c2e4307b8e36cae"
server_key_version: "encryptionkey@0"
I20260924 13:47:29.738116 18108 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.003s
I20260924 13:47:29.738881 18122 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 13:47:29.739068 18108 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 13:47:29.739187 18108 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790257629041645-13911-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790257629041645-13911-0/minicluster-data/ts-0/wal
uuid: "aff50161196a4d46ad09d8617159bf3b"
format_stamp: "Formatted at 2026-09-24 13:47:29 on dist-test-slave-sjl2"
server_key: "b6649d2e053061ff7266943a954e6e73"
server_key_iv: "794eb12aa77233b33c2e4307b8e36cae"
server_key_version: "encryptionkey@0"
I20260924 13:47:29.739254 18108 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790257629041645-13911-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790257629041645-13911-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790257629041645-13911-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 13:47:29.760241 18108 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 13:47:29.762380 18108 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 13:47:29.762532 18108 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 13:47:29.762784 18108 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 13:47:29.763089 18108 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 13:47:29.763123 18108 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 13:47:29.763152 18108 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 13:47:29.763176 18108 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 13:47:29.769062 18108 rpc_server.cc:307] RPC server started. Bound to: 127.13.149.193:38301
I20260924 13:47:29.769129 18236 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.149.193:38301 every 8 connection(s)
I20260924 13:47:29.769397 18108 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790257629041645-13911-0/minicluster-data/ts-0/data/info.pb
I20260924 13:47:29.771174 13911 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJN9WRF/build/release/bin/kudu as pid 18108
I20260924 13:47:29.771276 13911 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790257629041645-13911-0/minicluster-data/ts-0/wal/instance
I20260924 13:47:29.771436 13911 external_mini_cluster.cc:1428] Setting key 9c4eb7042f1a4bd5584cbe10bf644459
Sep 24 13:47:29 dist-test-slave-sjl2 krb5kdc[18001](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257649, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.149.193@KRBTEST.COM for kudu/127.13.149.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.13.149.194@KRBTEST.COM; defaulting to no policy
I20260924 13:47:29.780292 18086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:29.770151 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.149.193:52575 (local address 127.13.149.254:40811)
0924 13:47:29.770239 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0924 13:47:29.770242 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 13:47:29.770402 (+ 160us) server_negotiation.cc:407] Connection header received
0924 13:47:29.770917 (+ 515us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:47:29.770919 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:47:29.770931 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:47:29.770993 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:47:29.772142 (+ 1149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:29.772850 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:29.774004 (+ 1154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:29.774216 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:29.774281 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:47:29.777349 (+ 3068us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:47:29.777351 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:47:29.777352 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 13:47:29.777361 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:47:29.778757 (+ 1396us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:47:29.779322 (+ 565us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:47:29.779323 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:47:29.779323 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:47:29.779375 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:47:29.779662 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:47:29.779663 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:47:29.779664 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:47:29.779795 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:47:29.779849 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0924 13:47:29.780135 (+ 286us) server_negotiation.cc:299] Negotiation successful
0924 13:47:29.780211 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260924 13:47:29.780479 18239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:29.770111 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:40811 (local address 127.13.149.193:52575)
0924 13:47:29.770278 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0924 13:47:29.770318 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0924 13:47:29.770797 (+ 479us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:47:29.771062 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:47:29.771066 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:47:29.771395 (+ 329us) client_negotiation.cc:189] Negotiated authn=SASL
0924 13:47:29.772039 (+ 644us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:47:29.772048 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:47:29.772992 (+ 944us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:47:29.772993 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:47:29.773904 (+ 911us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:47:29.773907 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:47:29.774300 (+ 393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:47:29.774301 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:47:29.774452 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:47:29.775078 (+ 626us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 13:47:29.775093 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0924 13:47:29.777218 (+ 2125us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 13:47:29.778870 (+ 1652us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 13:47:29.778886 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 13:47:29.778889 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0924 13:47:29.779247 (+ 358us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 13:47:29.779432 (+ 185us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 13:47:29.779433 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 13:47:29.779433 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 13:47:29.779600 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 13:47:29.779861 (+ 261us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 13:47:29.779867 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 13:47:29.780014 (+ 147us) client_negotiation.cc:769] Sending connection context
0924 13:47:29.780113 (+ 99us) client_negotiation.cc:240] Negotiation successful
0924 13:47:29.780264 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":86,"threads_started":1}
I20260924 13:47:29.781091 18237 heartbeater.cc:347] Connected to a master server at 127.13.149.254:40811
I20260924 13:47:29.781200 18237 heartbeater.cc:464] Registering TS with master...
I20260924 13:47:29.781371 18237 heartbeater.cc:511] Master 127.13.149.254:40811 requested a full tablet report, sending...
I20260924 13:47:29.781879 18045 ts_manager.cc:194] Registered new tserver with Master: aff50161196a4d46ad09d8617159bf3b (127.13.149.193:38301)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.149.194@KRBTEST.COM" created.
I20260924 13:47:29.782541 18045 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.149.193@KRBTEST.COM'} at 127.13.149.193:52575
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.149.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.194.keytab.
Entry for principal kudu/127.13.149.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.194.keytab.
I20260924 13:47:29.797212 13911 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJN9WRF/build/release/bin/kudu
/tmp/dist-test-taskJN9WRF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790257629041645-13911-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790257629041645-13911-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790257629041645-13911-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790257629041645-13911-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.149.194:0
--local_ip_for_outbound_sockets=127.13.149.194
--webserver_interface=127.13.149.194
--webserver_port=0
--tserver_master_addrs=127.13.149.254:40811
--keytab_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.194.keytab
--principal=kudu/127.13.149.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.149.212:39509
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790257629041645-13911-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790257629041645-13911-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790257629041645-13911-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 13:47:29.881794 18242 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 13:47:29.882021 18242 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 13:47:29.882058 18242 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 13:47:29.883347 18242 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 13:47:29.883400 18242 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 13:47:29.883441 18242 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.149.194
I20260924 13:47:29.885231 18242 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.149.212:39509
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790257629041645-13911-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790257629041645-13911-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.194.keytab
--principal=kudu/127.13.149.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.149.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790257629041645-13911-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.149.194
--webserver_port=0
--tserver_master_addrs=127.13.149.254:40811
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18242
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.149.194
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790257629041645-13911-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 12:29:37 UTC on ccycloud.quasar-iqzbxn.root.comops.site
build id 4670106
I20260924 13:47:29.885509 18242 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 13:47:29.885751 18242 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 13:47:29.892050 18248 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 13:47:29.892091 18242 server_base.cc:1034] running on GCE node
W20260924 13:47:29.892053 18250 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 13:47:29.892087 18247 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 13:47:29.892503 18242 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 13:47:29.892771 18242 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 13:47:29.893954 18242 hybrid_clock.cc:648] HybridClock initialized: now 1790257649893935 us; error 35 us; skew 500 ppm
Sep 24 13:47:29 dist-test-slave-sjl2 krb5kdc[18001](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257649, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.149.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 13:47:29.896641 18242 init.cc:377] Logged in from keytab as kudu/127.13.149.194@KRBTEST.COM (short username kudu)
I20260924 13:47:29.897066 18242 fs_manager.cc:359] Metadata directory not provided
I20260924 13:47:29.897120 18242 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 13:47:29.897226 18242 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 13:47:29.898321 18242 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790257629041645-13911-0/minicluster-data/ts-1/data/instance:
uuid: "0b878d07b2d74f658d6f60629a7297b6"
format_stamp: "Formatted at 2026-09-24 13:47:29 on dist-test-slave-sjl2"
server_key: "5bc8364b1de7fc9c5b1e6e8d53b1f17a"
server_key_iv: "914818baff4c52591656ec23009781e9"
server_key_version: "encryptionkey@0"
I20260924 13:47:29.898643 18242 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790257629041645-13911-0/minicluster-data/ts-1/wal/instance:
uuid: "0b878d07b2d74f658d6f60629a7297b6"
format_stamp: "Formatted at 2026-09-24 13:47:29 on dist-test-slave-sjl2"
server_key: "5bc8364b1de7fc9c5b1e6e8d53b1f17a"
server_key_iv: "914818baff4c52591656ec23009781e9"
server_key_version: "encryptionkey@0"
I20260924 13:47:29.899899 18242 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260924 13:47:29.900588 18256 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 13:47:29.900734 18242 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 13:47:29.900807 18242 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790257629041645-13911-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790257629041645-13911-0/minicluster-data/ts-1/wal
uuid: "0b878d07b2d74f658d6f60629a7297b6"
format_stamp: "Formatted at 2026-09-24 13:47:29 on dist-test-slave-sjl2"
server_key: "5bc8364b1de7fc9c5b1e6e8d53b1f17a"
server_key_iv: "914818baff4c52591656ec23009781e9"
server_key_version: "encryptionkey@0"
I20260924 13:47:29.900885 18242 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790257629041645-13911-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790257629041645-13911-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790257629041645-13911-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 13:47:29.910969 18242 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 13:47:29.912954 18242 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 13:47:29.913101 18242 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 13:47:29.913487 18242 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 13:47:29.914007 18242 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 13:47:29.914049 18242 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 13:47:29.914073 18242 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 13:47:29.914084 18242 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 13:47:29.920034 18242 rpc_server.cc:307] RPC server started. Bound to: 127.13.149.194:33647
I20260924 13:47:29.920130 18370 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.149.194:33647 every 8 connection(s)
I20260924 13:47:29.920373 18242 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790257629041645-13911-0/minicluster-data/ts-1/data/info.pb
I20260924 13:47:29.922119 13911 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJN9WRF/build/release/bin/kudu as pid 18242
I20260924 13:47:29.922236 13911 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790257629041645-13911-0/minicluster-data/ts-1/wal/instance
I20260924 13:47:29.922431 13911 external_mini_cluster.cc:1428] Setting key 71e21c6137cdd6b6713444a7799bdb50
Sep 24 13:47:29 dist-test-slave-sjl2 krb5kdc[18001](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257649, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.149.194@KRBTEST.COM for kudu/127.13.149.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.13.149.195@KRBTEST.COM; defaulting to no policy
I20260924 13:47:29.931058 18373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:29.921088 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:40811 (local address 127.13.149.194:54999)
0924 13:47:29.921262 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0924 13:47:29.921303 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0924 13:47:29.921787 (+ 484us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:47:29.922029 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:47:29.922033 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:47:29.922421 (+ 388us) client_negotiation.cc:189] Negotiated authn=SASL
0924 13:47:29.923083 (+ 662us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:47:29.923092 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:47:29.923775 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:47:29.923777 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:47:29.924836 (+ 1059us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:47:29.924839 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:47:29.925372 (+ 533us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:47:29.925373 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:47:29.925495 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:47:29.926050 (+ 555us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 13:47:29.926059 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0924 13:47:29.927509 (+ 1450us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 13:47:29.929414 (+ 1905us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 13:47:29.929425 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 13:47:29.929427 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 13:47:29.929819 (+ 392us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 13:47:29.930038 (+ 219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 13:47:29.930039 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 13:47:29.930039 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 13:47:29.930166 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 13:47:29.930442 (+ 276us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 13:47:29.930446 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 13:47:29.930586 (+ 140us) client_negotiation.cc:769] Sending connection context
0924 13:47:29.930718 (+ 132us) client_negotiation.cc:240] Negotiation successful
0924 13:47:29.930865 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":88,"threads_started":1}
I20260924 13:47:29.931052 18086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:29.921092 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.149.194:54999 (local address 127.13.149.254:40811)
0924 13:47:29.921158 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0924 13:47:29.921160 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:47:29.921384 (+ 224us) server_negotiation.cc:407] Connection header received
0924 13:47:29.921876 (+ 492us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:47:29.921877 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:47:29.921887 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:47:29.921939 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:47:29.923169 (+ 1230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:29.923683 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:29.924966 (+ 1283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:29.925258 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:29.925337 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:47:29.927629 (+ 2292us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:47:29.927631 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:47:29.927631 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 13:47:29.927640 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:47:29.929303 (+ 1663us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:47:29.929905 (+ 602us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:47:29.929907 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:47:29.929907 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:47:29.929970 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:47:29.930230 (+ 260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:47:29.930231 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:47:29.930231 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:47:29.930383 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:47:29.930448 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0924 13:47:29.930771 (+ 323us) server_negotiation.cc:299] Negotiation successful
0924 13:47:29.930981 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260924 13:47:29.932291 18371 heartbeater.cc:347] Connected to a master server at 127.13.149.254:40811
I20260924 13:47:29.932430 18371 heartbeater.cc:464] Registering TS with master...
I20260924 13:47:29.932615 18371 heartbeater.cc:511] Master 127.13.149.254:40811 requested a full tablet report, sending...
I20260924 13:47:29.933153 18045 ts_manager.cc:194] Registered new tserver with Master: 0b878d07b2d74f658d6f60629a7297b6 (127.13.149.194:33647)
I20260924 13:47:29.933533 18045 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.149.194@KRBTEST.COM'} at 127.13.149.194:54999
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.149.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.149.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.195.keytab.
Entry for principal kudu/127.13.149.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.195.keytab.
I20260924 13:47:29.951808 13911 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJN9WRF/build/release/bin/kudu
/tmp/dist-test-taskJN9WRF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790257629041645-13911-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790257629041645-13911-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790257629041645-13911-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790257629041645-13911-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.149.195:0
--local_ip_for_outbound_sockets=127.13.149.195
--webserver_interface=127.13.149.195
--webserver_port=0
--tserver_master_addrs=127.13.149.254:40811
--keytab_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.195.keytab
--principal=kudu/127.13.149.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.149.212:39509
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790257629041645-13911-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790257629041645-13911-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790257629041645-13911-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 13:47:30.041468 18376 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 13:47:30.041698 18376 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 13:47:30.041730 18376 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 13:47:30.043061 18376 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 13:47:30.043118 18376 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 13:47:30.043149 18376 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.149.195
I20260924 13:47:30.044847 18376 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.149.212:39509
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790257629041645-13911-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790257629041645-13911-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.195.keytab
--principal=kudu/127.13.149.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.149.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790257629041645-13911-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.149.195
--webserver_port=0
--tserver_master_addrs=127.13.149.254:40811
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18376
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.149.195
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790257629041645-13911-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 12:29:37 UTC on ccycloud.quasar-iqzbxn.root.comops.site
build id 4670106
I20260924 13:47:30.045133 18376 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 13:47:30.045344 18376 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260924 13:47:30.051239 18376 server_base.cc:1034] running on GCE node
W20260924 13:47:30.051198 18381 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 13:47:30.051224 18382 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 13:47:30.051198 18384 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 13:47:30.051743 18376 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 13:47:30.052026 18376 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 13:47:30.053176 18376 hybrid_clock.cc:648] HybridClock initialized: now 1790257650053157 us; error 35 us; skew 500 ppm
Sep 24 13:47:30 dist-test-slave-sjl2 krb5kdc[18001](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257650, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.149.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 13:47:30.055871 18376 init.cc:377] Logged in from keytab as kudu/127.13.149.195@KRBTEST.COM (short username kudu)
I20260924 13:47:30.056237 18376 fs_manager.cc:359] Metadata directory not provided
I20260924 13:47:30.056290 18376 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 13:47:30.056433 18376 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 13:47:30.057763 18376 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790257629041645-13911-0/minicluster-data/ts-2/data/instance:
uuid: "e8be7acb295b4500844d25609fb0bb8d"
format_stamp: "Formatted at 2026-09-24 13:47:30 on dist-test-slave-sjl2"
server_key: "f9b60095d267691a1ef751fb61843b7a"
server_key_iv: "b6b0c28c7a45109f80cb8f8170ed5de3"
server_key_version: "encryptionkey@0"
I20260924 13:47:30.058158 18376 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790257629041645-13911-0/minicluster-data/ts-2/wal/instance:
uuid: "e8be7acb295b4500844d25609fb0bb8d"
format_stamp: "Formatted at 2026-09-24 13:47:30 on dist-test-slave-sjl2"
server_key: "f9b60095d267691a1ef751fb61843b7a"
server_key_iv: "b6b0c28c7a45109f80cb8f8170ed5de3"
server_key_version: "encryptionkey@0"
I20260924 13:47:30.059515 18376 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260924 13:47:30.060415 18390 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 13:47:30.060581 18376 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260924 13:47:30.060724 18376 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790257629041645-13911-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790257629041645-13911-0/minicluster-data/ts-2/wal
uuid: "e8be7acb295b4500844d25609fb0bb8d"
format_stamp: "Formatted at 2026-09-24 13:47:30 on dist-test-slave-sjl2"
server_key: "f9b60095d267691a1ef751fb61843b7a"
server_key_iv: "b6b0c28c7a45109f80cb8f8170ed5de3"
server_key_version: "encryptionkey@0"
I20260924 13:47:30.060798 18376 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790257629041645-13911-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790257629041645-13911-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790257629041645-13911-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 13:47:30.072399 18376 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 13:47:30.075060 18376 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 13:47:30.075218 18376 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 13:47:30.075445 18376 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 13:47:30.075788 18376 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 13:47:30.075824 18376 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 13:47:30.075863 18376 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 13:47:30.075888 18376 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 13:47:30.081764 18376 rpc_server.cc:307] RPC server started. Bound to: 127.13.149.195:34105
I20260924 13:47:30.081984 18504 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.149.195:34105 every 8 connection(s)
I20260924 13:47:30.082114 18376 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790257629041645-13911-0/minicluster-data/ts-2/data/info.pb
I20260924 13:47:30.087538 13911 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJN9WRF/build/release/bin/kudu as pid 18376
I20260924 13:47:30.087656 13911 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790257629041645-13911-0/minicluster-data/ts-2/wal/instance
I20260924 13:47:30.087899 13911 external_mini_cluster.cc:1428] Setting key d39c2abff84d433034dd7bd14bae1150
Sep 24 13:47:30 dist-test-slave-sjl2 krb5kdc[18001](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257650, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.149.195@KRBTEST.COM for kudu/127.13.149.254@KRBTEST.COM
I20260924 13:47:30.091925 18086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:30.082922 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.149.195:60437 (local address 127.13.149.254:40811)
0924 13:47:30.082986 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0924 13:47:30.082988 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:47:30.083222 (+ 234us) server_negotiation.cc:407] Connection header received
0924 13:47:30.083781 (+ 559us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:47:30.083783 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:47:30.083798 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:47:30.083847 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:47:30.084898 (+ 1051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:30.085441 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:30.086590 (+ 1149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:30.086821 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:30.086889 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:47:30.089314 (+ 2425us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:47:30.089316 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:47:30.089316 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 13:47:30.089323 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:47:30.090706 (+ 1383us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:47:30.091069 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:47:30.091070 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:47:30.091070 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:47:30.091118 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:47:30.091343 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:47:30.091345 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:47:30.091345 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:47:30.091484 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:47:30.091550 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0924 13:47:30.091777 (+ 227us) server_negotiation.cc:299] Negotiation successful
0924 13:47:30.091856 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260924 13:47:30.092065 18507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:30.082883 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:40811 (local address 127.13.149.195:60437)
0924 13:47:30.083058 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0924 13:47:30.083135 (+ 77us) client_negotiation.cc:174] Beginning negotiation
0924 13:47:30.083666 (+ 531us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:47:30.083934 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:47:30.083938 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:47:30.084241 (+ 303us) client_negotiation.cc:189] Negotiated authn=SASL
0924 13:47:30.084784 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:47:30.084790 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:47:30.085539 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:47:30.085540 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:47:30.086491 (+ 951us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:47:30.086494 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:47:30.086916 (+ 422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:47:30.086917 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:47:30.087059 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:47:30.087671 (+ 612us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 13:47:30.087682 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0924 13:47:30.089212 (+ 1530us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 13:47:30.090782 (+ 1570us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 13:47:30.090789 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 13:47:30.090791 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 13:47:30.091010 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 13:47:30.091165 (+ 155us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 13:47:30.091165 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 13:47:30.091165 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 13:47:30.091269 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 13:47:30.091543 (+ 274us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 13:47:30.091546 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 13:47:30.091669 (+ 123us) client_negotiation.cc:769] Sending connection context
0924 13:47:30.091748 (+ 79us) client_negotiation.cc:240] Negotiation successful
0924 13:47:30.091872 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":87,"threads_started":1}
I20260924 13:47:30.092522 18505 heartbeater.cc:347] Connected to a master server at 127.13.149.254:40811
I20260924 13:47:30.092617 18505 heartbeater.cc:464] Registering TS with master...
I20260924 13:47:30.092737 18505 heartbeater.cc:511] Master 127.13.149.254:40811 requested a full tablet report, sending...
I20260924 13:47:30.093101 18045 ts_manager.cc:194] Registered new tserver with Master: e8be7acb295b4500844d25609fb0bb8d (127.13.149.195:34105)
I20260924 13:47:30.093466 18045 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.149.195@KRBTEST.COM'} at 127.13.149.195:60437
I20260924 13:47:30.099715 13911 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260924 13:47:30.107215 18086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:30.101097 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48990 (local address 127.13.149.254:40811)
0924 13:47:30.101171 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0924 13:47:30.101173 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:47:30.101231 (+ 58us) server_negotiation.cc:407] Connection header received
0924 13:47:30.101302 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:47:30.101302 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:47:30.101312 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:47:30.101347 (+ 35us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:47:30.101942 (+ 595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:30.102428 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:30.103252 (+ 824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:30.103470 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:30.103523 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:47:30.104796 (+ 1273us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:47:30.104797 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:47:30.104797 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 13:47:30.104804 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:47:30.106146 (+ 1342us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:47:30.106476 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:47:30.106477 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:47:30.106477 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:47:30.106533 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:47:30.106726 (+ 193us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:47:30.106726 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:47:30.106726 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:47:30.106902 (+ 176us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:47:30.106976 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0924 13:47:30.107078 (+ 102us) server_negotiation.cc:299] Negotiation successful
0924 13:47:30.107143 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260924 13:47:30.109215 18045 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48990:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 13:47:30.111613 18045 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 13:47:30.118770 18520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:30.114667 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54748 (local address 127.13.149.193:38301)
0924 13:47:30.114903 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0924 13:47:30.114906 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 13:47:30.114917 (+ 11us) server_negotiation.cc:407] Connection header received
0924 13:47:30.114963 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:47:30.114965 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:47:30.114982 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:47:30.115090 (+ 108us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 13:47:30.115303 (+ 213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:30.116149 (+ 846us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:30.117546 (+ 1397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:30.118263 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:30.118349 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:47:30.118411 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0924 13:47:30.118614 (+ 203us) server_negotiation.cc:299] Negotiation successful
0924 13:47:30.118676 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":204,"thread_start_us":84,"threads_started":1}
I20260924 13:47:30.118762 18518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:30.114447 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.193:38301 (local address 127.0.0.1:54748)
0924 13:47:30.114660 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0924 13:47:30.114706 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0924 13:47:30.114781 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:47:30.115099 (+ 318us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:47:30.115103 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:47:30.115106 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 13:47:30.115207 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:47:30.115209 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:47:30.116253 (+ 1044us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:47:30.116254 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:47:30.117451 (+ 1197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:47:30.117455 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:47:30.118363 (+ 908us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:47:30.118364 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:47:30.118478 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:47:30.118495 (+ 17us) client_negotiation.cc:769] Sending connection context
0924 13:47:30.118591 (+ 96us) client_negotiation.cc:240] Negotiation successful
0924 13:47:30.118663 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":92,"threads_started":1}
I20260924 13:47:30.118795 18519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:30.114593 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37534 (local address 127.13.149.194:33647)
0924 13:47:30.114824 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0924 13:47:30.114829 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 13:47:30.114981 (+ 152us) server_negotiation.cc:407] Connection header received
0924 13:47:30.115072 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:47:30.115074 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:47:30.115089 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:47:30.115185 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 13:47:30.115409 (+ 224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:30.116507 (+ 1098us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:30.117719 (+ 1212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:30.118322 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:30.118406 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:47:30.118467 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0924 13:47:30.118675 (+ 208us) server_negotiation.cc:299] Negotiation successful
0924 13:47:30.118720 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":187,"thread_start_us":137,"threads_started":1}
I20260924 13:47:30.119011 18516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:30.114447 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.194:33647 (local address 127.0.0.1:37534)
0924 13:47:30.114920 (+ 473us) negotiation.cc:107] Waiting for socket to connect
0924 13:47:30.114930 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 13:47:30.115005 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:47:30.115276 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:47:30.115277 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:47:30.115280 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 13:47:30.115328 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:47:30.115330 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:47:30.116627 (+ 1297us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:47:30.116628 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:47:30.117627 (+ 999us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:47:30.117630 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:47:30.118504 (+ 874us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:47:30.118504 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:47:30.118570 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:47:30.118576 (+ 6us) client_negotiation.cc:769] Sending connection context
0924 13:47:30.118883 (+ 307us) client_negotiation.cc:240] Negotiation successful
0924 13:47:30.118937 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":464,"mutex_wait_us":347,"thread_start_us":79,"threads_started":1}
I20260924 13:47:30.119377 18522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:30.114842 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46030 (local address 127.13.149.195:34105)
0924 13:47:30.115229 (+ 387us) server_negotiation.cc:206] Beginning negotiation
0924 13:47:30.115233 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 13:47:30.115243 (+ 10us) server_negotiation.cc:407] Connection header received
0924 13:47:30.115284 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:47:30.115285 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:47:30.115301 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:47:30.115377 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 13:47:30.115594 (+ 217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:30.116657 (+ 1063us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:30.117923 (+ 1266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:30.118605 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:30.118674 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:47:30.118721 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0924 13:47:30.119236 (+ 515us) server_negotiation.cc:299] Negotiation successful
0924 13:47:30.119291 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":351,"thread_start_us":74,"threads_started":1}
I20260924 13:47:30.119601 18521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:30.114447 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.195:34105 (local address 127.0.0.1:46030)
0924 13:47:30.114911 (+ 464us) negotiation.cc:107] Waiting for socket to connect
0924 13:47:30.114919 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0924 13:47:30.114960 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:47:30.115465 (+ 505us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:47:30.115466 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:47:30.115468 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 13:47:30.115508 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:47:30.115510 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:47:30.116774 (+ 1264us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:47:30.116776 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:47:30.117827 (+ 1051us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:47:30.117831 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:47:30.118758 (+ 927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:47:30.118759 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:47:30.118823 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:47:30.118828 (+ 5us) client_negotiation.cc:769] Sending connection context
0924 13:47:30.118876 (+ 48us) client_negotiation.cc:240] Negotiation successful
0924 13:47:30.119525 (+ 649us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":445,"mutex_wait_us":2,"thread_start_us":84,"threads_started":1}
I20260924 13:47:30.119855 18304 tablet_service.cc:1467] Processing CreateTablet for tablet d668e00555be4e298a5ad1a09670744e (DEFAULT_TABLE table=test-table [id=c823429b2e214ba185df62c7db28adf3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 13:47:30.120292 18304 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d668e00555be4e298a5ad1a09670744e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 13:47:30.120283 18438 tablet_service.cc:1467] Processing CreateTablet for tablet d668e00555be4e298a5ad1a09670744e (DEFAULT_TABLE table=test-table [id=c823429b2e214ba185df62c7db28adf3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 13:47:30.120620 18438 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d668e00555be4e298a5ad1a09670744e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 13:47:30.124115 18523 tablet_bootstrap.cc:492] T d668e00555be4e298a5ad1a09670744e P 0b878d07b2d74f658d6f60629a7297b6: Bootstrap starting.
I20260924 13:47:30.124534 18170 tablet_service.cc:1467] Processing CreateTablet for tablet d668e00555be4e298a5ad1a09670744e (DEFAULT_TABLE table=test-table [id=c823429b2e214ba185df62c7db28adf3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 13:47:30.124883 18170 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d668e00555be4e298a5ad1a09670744e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 13:47:30.124956 18523 tablet_bootstrap.cc:654] T d668e00555be4e298a5ad1a09670744e P 0b878d07b2d74f658d6f60629a7297b6: Neither blocks nor log segments found. Creating new log.
I20260924 13:47:30.125629 18523 log.cc:824] T d668e00555be4e298a5ad1a09670744e P 0b878d07b2d74f658d6f60629a7297b6: Log is configured to *not* fsync() on all Append() calls
I20260924 13:47:30.126272 18524 tablet_bootstrap.cc:492] T d668e00555be4e298a5ad1a09670744e P e8be7acb295b4500844d25609fb0bb8d: Bootstrap starting.
I20260924 13:47:30.126893 18524 tablet_bootstrap.cc:654] T d668e00555be4e298a5ad1a09670744e P e8be7acb295b4500844d25609fb0bb8d: Neither blocks nor log segments found. Creating new log.
I20260924 13:47:30.127384 18524 log.cc:824] T d668e00555be4e298a5ad1a09670744e P e8be7acb295b4500844d25609fb0bb8d: Log is configured to *not* fsync() on all Append() calls
I20260924 13:47:30.127995 18527 tablet_bootstrap.cc:492] T d668e00555be4e298a5ad1a09670744e P aff50161196a4d46ad09d8617159bf3b: Bootstrap starting.
I20260924 13:47:30.128173 18523 tablet_bootstrap.cc:492] T d668e00555be4e298a5ad1a09670744e P 0b878d07b2d74f658d6f60629a7297b6: No bootstrap required, opened a new log
I20260924 13:47:30.128228 18524 tablet_bootstrap.cc:492] T d668e00555be4e298a5ad1a09670744e P e8be7acb295b4500844d25609fb0bb8d: No bootstrap required, opened a new log
I20260924 13:47:30.128254 18523 ts_tablet_manager.cc:1397] T d668e00555be4e298a5ad1a09670744e P 0b878d07b2d74f658d6f60629a7297b6: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260924 13:47:30.128299 18524 ts_tablet_manager.cc:1397] T d668e00555be4e298a5ad1a09670744e P e8be7acb295b4500844d25609fb0bb8d: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260924 13:47:30.128739 18527 tablet_bootstrap.cc:654] T d668e00555be4e298a5ad1a09670744e P aff50161196a4d46ad09d8617159bf3b: Neither blocks nor log segments found. Creating new log.
I20260924 13:47:30.129381 18527 log.cc:824] T d668e00555be4e298a5ad1a09670744e P aff50161196a4d46ad09d8617159bf3b: Log is configured to *not* fsync() on all Append() calls
I20260924 13:47:30.130311 18527 tablet_bootstrap.cc:492] T d668e00555be4e298a5ad1a09670744e P aff50161196a4d46ad09d8617159bf3b: No bootstrap required, opened a new log
I20260924 13:47:30.130175 18524 raft_consensus.cc:348] T d668e00555be4e298a5ad1a09670744e P e8be7acb295b4500844d25609fb0bb8d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aff50161196a4d46ad09d8617159bf3b" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 38301 } } peers { permanent_uuid: "e8be7acb295b4500844d25609fb0bb8d" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 34105 } } peers { permanent_uuid: "0b878d07b2d74f658d6f60629a7297b6" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 33647 } }
I20260924 13:47:30.130360 18524 raft_consensus.cc:374] T d668e00555be4e298a5ad1a09670744e P e8be7acb295b4500844d25609fb0bb8d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 13:47:30.130388 18527 ts_tablet_manager.cc:1397] T d668e00555be4e298a5ad1a09670744e P aff50161196a4d46ad09d8617159bf3b: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260924 13:47:30.130399 18524 raft_consensus.cc:729] T d668e00555be4e298a5ad1a09670744e P e8be7acb295b4500844d25609fb0bb8d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8be7acb295b4500844d25609fb0bb8d, State: Initialized, Role: FOLLOWER
I20260924 13:47:30.130339 18523 raft_consensus.cc:348] T d668e00555be4e298a5ad1a09670744e P 0b878d07b2d74f658d6f60629a7297b6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aff50161196a4d46ad09d8617159bf3b" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 38301 } } peers { permanent_uuid: "e8be7acb295b4500844d25609fb0bb8d" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 34105 } } peers { permanent_uuid: "0b878d07b2d74f658d6f60629a7297b6" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 33647 } }
I20260924 13:47:30.130504 18523 raft_consensus.cc:374] T d668e00555be4e298a5ad1a09670744e P 0b878d07b2d74f658d6f60629a7297b6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 13:47:30.130537 18523 raft_consensus.cc:729] T d668e00555be4e298a5ad1a09670744e P 0b878d07b2d74f658d6f60629a7297b6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0b878d07b2d74f658d6f60629a7297b6, State: Initialized, Role: FOLLOWER
I20260924 13:47:30.130576 18524 consensus_queue.cc:260] T d668e00555be4e298a5ad1a09670744e P e8be7acb295b4500844d25609fb0bb8d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aff50161196a4d46ad09d8617159bf3b" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 38301 } } peers { permanent_uuid: "e8be7acb295b4500844d25609fb0bb8d" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 34105 } } peers { permanent_uuid: "0b878d07b2d74f658d6f60629a7297b6" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 33647 } }
I20260924 13:47:30.130678 18523 consensus_queue.cc:260] T d668e00555be4e298a5ad1a09670744e P 0b878d07b2d74f658d6f60629a7297b6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aff50161196a4d46ad09d8617159bf3b" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 38301 } } peers { permanent_uuid: "e8be7acb295b4500844d25609fb0bb8d" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 34105 } } peers { permanent_uuid: "0b878d07b2d74f658d6f60629a7297b6" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 33647 } }
I20260924 13:47:30.130899 18524 ts_tablet_manager.cc:1428] T d668e00555be4e298a5ad1a09670744e P e8be7acb295b4500844d25609fb0bb8d: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260924 13:47:30.130952 18523 ts_tablet_manager.cc:1428] T d668e00555be4e298a5ad1a09670744e P 0b878d07b2d74f658d6f60629a7297b6: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260924 13:47:30.131029 18505 heartbeater.cc:503] Master 127.13.149.254:40811 was elected leader, sending a full tablet report...
I20260924 13:47:30.131060 18371 heartbeater.cc:503] Master 127.13.149.254:40811 was elected leader, sending a full tablet report...
I20260924 13:47:30.132445 18527 raft_consensus.cc:348] T d668e00555be4e298a5ad1a09670744e P aff50161196a4d46ad09d8617159bf3b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aff50161196a4d46ad09d8617159bf3b" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 38301 } } peers { permanent_uuid: "e8be7acb295b4500844d25609fb0bb8d" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 34105 } } peers { permanent_uuid: "0b878d07b2d74f658d6f60629a7297b6" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 33647 } }
I20260924 13:47:30.132606 18527 raft_consensus.cc:374] T d668e00555be4e298a5ad1a09670744e P aff50161196a4d46ad09d8617159bf3b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 13:47:30.132628 18527 raft_consensus.cc:729] T d668e00555be4e298a5ad1a09670744e P aff50161196a4d46ad09d8617159bf3b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aff50161196a4d46ad09d8617159bf3b, State: Initialized, Role: FOLLOWER
I20260924 13:47:30.132726 18527 consensus_queue.cc:260] T d668e00555be4e298a5ad1a09670744e P aff50161196a4d46ad09d8617159bf3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aff50161196a4d46ad09d8617159bf3b" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 38301 } } peers { permanent_uuid: "e8be7acb295b4500844d25609fb0bb8d" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 34105 } } peers { permanent_uuid: "0b878d07b2d74f658d6f60629a7297b6" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 33647 } }
I20260924 13:47:30.133038 18527 ts_tablet_manager.cc:1428] T d668e00555be4e298a5ad1a09670744e P aff50161196a4d46ad09d8617159bf3b: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260924 13:47:30.133100 18237 heartbeater.cc:503] Master 127.13.149.254:40811 was elected leader, sending a full tablet report...
W20260924 13:47:30.170941 18372 tablet.cc:2369] T d668e00555be4e298a5ad1a09670744e P 0b878d07b2d74f658d6f60629a7297b6: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260924 13:47:30.201316 18531 raft_consensus.cc:482] T d668e00555be4e298a5ad1a09670744e P aff50161196a4d46ad09d8617159bf3b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 13:47:30.201453 18531 raft_consensus.cc:504] T d668e00555be4e298a5ad1a09670744e P aff50161196a4d46ad09d8617159bf3b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aff50161196a4d46ad09d8617159bf3b" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 38301 } } peers { permanent_uuid: "e8be7acb295b4500844d25609fb0bb8d" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 34105 } } peers { permanent_uuid: "0b878d07b2d74f658d6f60629a7297b6" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 33647 } }
I20260924 13:47:30.201789 18531 leader_election.cc:288] T d668e00555be4e298a5ad1a09670744e P aff50161196a4d46ad09d8617159bf3b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8be7acb295b4500844d25609fb0bb8d (127.13.149.195:34105), 0b878d07b2d74f658d6f60629a7297b6 (127.13.149.194:33647)
I20260924 13:47:30.205734 18522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:30.202024 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.149.193:50351 (local address 127.13.149.195:34105)
0924 13:47:30.202061 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0924 13:47:30.202063 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:47:30.202072 (+ 9us) server_negotiation.cc:407] Connection header received
0924 13:47:30.202170 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:47:30.202172 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:47:30.202180 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:47:30.202236 (+ 56us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 13:47:30.202372 (+ 136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:30.203130 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:30.204719 (+ 1589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:30.205340 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:30.205408 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:47:30.205417 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0924 13:47:30.205617 (+ 200us) server_negotiation.cc:299] Negotiation successful
0924 13:47:30.205660 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260924 13:47:30.205760 18519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:30.202039 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.149.193:33843 (local address 127.13.149.194:33647)
0924 13:47:30.202082 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0924 13:47:30.202085 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 13:47:30.202227 (+ 142us) server_negotiation.cc:407] Connection header received
0924 13:47:30.202259 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:47:30.202260 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:47:30.202270 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:47:30.202316 (+ 46us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 13:47:30.202446 (+ 130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:30.203150 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:30.204719 (+ 1569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:30.205326 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:30.205408 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:47:30.205417 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0924 13:47:30.205665 (+ 248us) server_negotiation.cc:299] Negotiation successful
0924 13:47:30.205707 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260924 13:47:30.205752 18239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:30.201968 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.195:34105 (local address 127.13.149.193:50351)
0924 13:47:30.202024 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0924 13:47:30.202036 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0924 13:47:30.202096 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:47:30.202249 (+ 153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:47:30.202249 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:47:30.202251 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 13:47:30.202306 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:47:30.202308 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:47:30.203236 (+ 928us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:47:30.203237 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:47:30.204608 (+ 1371us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:47:30.204612 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:47:30.205440 (+ 828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:47:30.205441 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:47:30.205536 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:47:30.205543 (+ 7us) client_negotiation.cc:769] Sending connection context
0924 13:47:30.205610 (+ 67us) client_negotiation.cc:240] Negotiation successful
0924 13:47:30.205658 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36,"mutex_wait_us":33}
I20260924 13:47:30.205761 18532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:30.201979 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.194:33647 (local address 127.13.149.193:33843)
0924 13:47:30.202164 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0924 13:47:30.202173 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0924 13:47:30.202220 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:47:30.202335 (+ 115us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:47:30.202336 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:47:30.202337 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 13:47:30.202394 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:47:30.202395 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:47:30.203236 (+ 841us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:47:30.203237 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:47:30.204645 (+ 1408us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:47:30.204647 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:47:30.205440 (+ 793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:47:30.205441 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:47:30.205536 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:47:30.205543 (+ 7us) client_negotiation.cc:769] Sending connection context
0924 13:47:30.205610 (+ 67us) client_negotiation.cc:240] Negotiation successful
0924 13:47:30.205679 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":169,"mutex_wait_us":64,"thread_start_us":105,"threads_started":1}
I20260924 13:47:30.206159 18458 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d668e00555be4e298a5ad1a09670744e" candidate_uuid: "aff50161196a4d46ad09d8617159bf3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8be7acb295b4500844d25609fb0bb8d" is_pre_election: true
I20260924 13:47:30.206310 18458 raft_consensus.cc:2454] T d668e00555be4e298a5ad1a09670744e P e8be7acb295b4500844d25609fb0bb8d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aff50161196a4d46ad09d8617159bf3b in term 0.
I20260924 13:47:30.206364 18325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d668e00555be4e298a5ad1a09670744e" candidate_uuid: "aff50161196a4d46ad09d8617159bf3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0b878d07b2d74f658d6f60629a7297b6" is_pre_election: true
I20260924 13:47:30.206518 18325 raft_consensus.cc:2454] T d668e00555be4e298a5ad1a09670744e P 0b878d07b2d74f658d6f60629a7297b6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aff50161196a4d46ad09d8617159bf3b in term 0.
I20260924 13:47:30.206565 18125 leader_election.cc:302] T d668e00555be4e298a5ad1a09670744e P aff50161196a4d46ad09d8617159bf3b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aff50161196a4d46ad09d8617159bf3b, e8be7acb295b4500844d25609fb0bb8d; no voters:
I20260924 13:47:30.206696 18531 raft_consensus.cc:2781] T d668e00555be4e298a5ad1a09670744e P aff50161196a4d46ad09d8617159bf3b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 13:47:30.206763 18531 raft_consensus.cc:482] T d668e00555be4e298a5ad1a09670744e P aff50161196a4d46ad09d8617159bf3b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 13:47:30.206785 18531 raft_consensus.cc:3037] T d668e00555be4e298a5ad1a09670744e P aff50161196a4d46ad09d8617159bf3b [term 0 FOLLOWER]: Advancing to term 1
I20260924 13:47:30.207471 18531 raft_consensus.cc:504] T d668e00555be4e298a5ad1a09670744e P aff50161196a4d46ad09d8617159bf3b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aff50161196a4d46ad09d8617159bf3b" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 38301 } } peers { permanent_uuid: "e8be7acb295b4500844d25609fb0bb8d" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 34105 } } peers { permanent_uuid: "0b878d07b2d74f658d6f60629a7297b6" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 33647 } }
I20260924 13:47:30.207696 18531 leader_election.cc:288] T d668e00555be4e298a5ad1a09670744e P aff50161196a4d46ad09d8617159bf3b [CANDIDATE]: Term 1 election: Requested vote from peers e8be7acb295b4500844d25609fb0bb8d (127.13.149.195:34105), 0b878d07b2d74f658d6f60629a7297b6 (127.13.149.194:33647)
I20260924 13:47:30.207877 18458 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d668e00555be4e298a5ad1a09670744e" candidate_uuid: "aff50161196a4d46ad09d8617159bf3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8be7acb295b4500844d25609fb0bb8d"
I20260924 13:47:30.207950 18325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d668e00555be4e298a5ad1a09670744e" candidate_uuid: "aff50161196a4d46ad09d8617159bf3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0b878d07b2d74f658d6f60629a7297b6"
I20260924 13:47:30.207962 18458 raft_consensus.cc:3037] T d668e00555be4e298a5ad1a09670744e P e8be7acb295b4500844d25609fb0bb8d [term 0 FOLLOWER]: Advancing to term 1
I20260924 13:47:30.208031 18325 raft_consensus.cc:3037] T d668e00555be4e298a5ad1a09670744e P 0b878d07b2d74f658d6f60629a7297b6 [term 0 FOLLOWER]: Advancing to term 1
I20260924 13:47:30.208681 18458 raft_consensus.cc:2454] T d668e00555be4e298a5ad1a09670744e P e8be7acb295b4500844d25609fb0bb8d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aff50161196a4d46ad09d8617159bf3b in term 1.
I20260924 13:47:30.208721 18325 raft_consensus.cc:2454] T d668e00555be4e298a5ad1a09670744e P 0b878d07b2d74f658d6f60629a7297b6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aff50161196a4d46ad09d8617159bf3b in term 1.
I20260924 13:47:30.208868 18125 leader_election.cc:302] T d668e00555be4e298a5ad1a09670744e P aff50161196a4d46ad09d8617159bf3b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aff50161196a4d46ad09d8617159bf3b, e8be7acb295b4500844d25609fb0bb8d; no voters:
I20260924 13:47:30.209056 18531 raft_consensus.cc:2781] T d668e00555be4e298a5ad1a09670744e P aff50161196a4d46ad09d8617159bf3b [term 1 FOLLOWER]: Leader election won for term 1
I20260924 13:47:30.209250 18531 raft_consensus.cc:686] T d668e00555be4e298a5ad1a09670744e P aff50161196a4d46ad09d8617159bf3b [term 1 LEADER]: Becoming Leader. State: Replica: aff50161196a4d46ad09d8617159bf3b, State: Running, Role: LEADER
I20260924 13:47:30.209352 18531 consensus_queue.cc:237] T d668e00555be4e298a5ad1a09670744e P aff50161196a4d46ad09d8617159bf3b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aff50161196a4d46ad09d8617159bf3b" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 38301 } } peers { permanent_uuid: "e8be7acb295b4500844d25609fb0bb8d" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 34105 } } peers { permanent_uuid: "0b878d07b2d74f658d6f60629a7297b6" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 33647 } }
I20260924 13:47:30.210243 18044 catalog_manager.cc:5495] T d668e00555be4e298a5ad1a09670744e P aff50161196a4d46ad09d8617159bf3b reported cstate change: term changed from 0 to 1, leader changed from <none> to aff50161196a4d46ad09d8617159bf3b (127.13.149.193). New cstate: current_term: 1 leader_uuid: "aff50161196a4d46ad09d8617159bf3b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aff50161196a4d46ad09d8617159bf3b" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 38301 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e8be7acb295b4500844d25609fb0bb8d" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 34105 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0b878d07b2d74f658d6f60629a7297b6" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 33647 } health_report { overall_health: UNKNOWN } } }
I20260924 13:47:30.236793 18520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:30.234338 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54758 (local address 127.13.149.193:38301)
0924 13:47:30.234396 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0924 13:47:30.234398 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:47:30.234408 (+ 10us) server_negotiation.cc:407] Connection header received
0924 13:47:30.234450 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:47:30.234451 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:47:30.234472 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:47:30.234537 (+ 65us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 13:47:30.234646 (+ 109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:30.235124 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:30.236021 (+ 897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:30.236225 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:30.236288 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:47:30.236458 (+ 170us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 13:47:30.236568 (+ 110us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 13:47:30.236653 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0924 13:47:30.236666 (+ 13us) server_negotiation.cc:299] Negotiation successful
0924 13:47:30.236689 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260924 13:47:30.238623 18458 raft_consensus.cc:1261] T d668e00555be4e298a5ad1a09670744e P e8be7acb295b4500844d25609fb0bb8d [term 1 FOLLOWER]: Refusing update from remote peer aff50161196a4d46ad09d8617159bf3b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 13:47:30.238624 18325 raft_consensus.cc:1261] T d668e00555be4e298a5ad1a09670744e P 0b878d07b2d74f658d6f60629a7297b6 [term 1 FOLLOWER]: Refusing update from remote peer aff50161196a4d46ad09d8617159bf3b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 13:47:30.239041 18533 consensus_queue.cc:1035] T d668e00555be4e298a5ad1a09670744e P aff50161196a4d46ad09d8617159bf3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e8be7acb295b4500844d25609fb0bb8d" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 34105 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 13:47:30.239188 18531 consensus_queue.cc:1035] T d668e00555be4e298a5ad1a09670744e P aff50161196a4d46ad09d8617159bf3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "0b878d07b2d74f658d6f60629a7297b6" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 33647 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 13:47:30.244129 18045 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48990:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 13:47:30.244278 18045 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48990:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 13:47:30.245100 18045 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3022da7e5ed242b29d69bd84984ac2ab: Sending DeleteTablet for 3 replicas of tablet d668e00555be4e298a5ad1a09670744e
I20260924 13:47:30.245548 18438 tablet_service.cc:1514] Processing DeleteTablet for tablet d668e00555be4e298a5ad1a09670744e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 13:47:30 UTC) from {username='kudu', principal='kudu/127.13.149.254@KRBTEST.COM'} at 127.0.0.1:46030
I20260924 13:47:30.245582 18304 tablet_service.cc:1514] Processing DeleteTablet for tablet d668e00555be4e298a5ad1a09670744e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 13:47:30 UTC) from {username='kudu', principal='kudu/127.13.149.254@KRBTEST.COM'} at 127.0.0.1:37534
I20260924 13:47:30.245823 18547 tablet_replica.cc:330] T d668e00555be4e298a5ad1a09670744e P 0b878d07b2d74f658d6f60629a7297b6: stopping tablet replica
I20260924 13:47:30.245865 18170 tablet_service.cc:1514] Processing DeleteTablet for tablet d668e00555be4e298a5ad1a09670744e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 13:47:30 UTC) from {username='kudu', principal='kudu/127.13.149.254@KRBTEST.COM'} at 127.0.0.1:54748
I20260924 13:47:30.245931 18547 raft_consensus.cc:2229] T d668e00555be4e298a5ad1a09670744e P 0b878d07b2d74f658d6f60629a7297b6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 13:47:30.246047 18547 raft_consensus.cc:2258] T d668e00555be4e298a5ad1a09670744e P 0b878d07b2d74f658d6f60629a7297b6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 13:47:30.246153 18548 tablet_replica.cc:330] T d668e00555be4e298a5ad1a09670744e P aff50161196a4d46ad09d8617159bf3b: stopping tablet replica
I20260924 13:47:30.246258 18548 raft_consensus.cc:2229] T d668e00555be4e298a5ad1a09670744e P aff50161196a4d46ad09d8617159bf3b [term 1 LEADER]: Raft consensus shutting down.
I20260924 13:47:30.246261 13911 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJN9WRF/build/release/bin/kudu with pid 18108
I20260924 13:47:30.246354 18548 raft_consensus.cc:2258] T d668e00555be4e298a5ad1a09670744e P aff50161196a4d46ad09d8617159bf3b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 13:47:30.246552 18547 ts_tablet_manager.cc:1905] T d668e00555be4e298a5ad1a09670744e P 0b878d07b2d74f658d6f60629a7297b6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 13:47:30.246949 18546 tablet_replica.cc:330] T d668e00555be4e298a5ad1a09670744e P e8be7acb295b4500844d25609fb0bb8d: stopping tablet replica
I20260924 13:47:30.247056 18546 raft_consensus.cc:2229] T d668e00555be4e298a5ad1a09670744e P e8be7acb295b4500844d25609fb0bb8d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 13:47:30.247162 18546 raft_consensus.cc:2258] T d668e00555be4e298a5ad1a09670744e P e8be7acb295b4500844d25609fb0bb8d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 13:47:30.247630 18546 ts_tablet_manager.cc:1905] T d668e00555be4e298a5ad1a09670744e P e8be7acb295b4500844d25609fb0bb8d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 13:47:30.248337 18547 ts_tablet_manager.cc:1918] T d668e00555be4e298a5ad1a09670744e P 0b878d07b2d74f658d6f60629a7297b6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 13:47:30.248399 18547 log.cc:1196] T d668e00555be4e298a5ad1a09670744e P 0b878d07b2d74f658d6f60629a7297b6: Deleting WAL directory at /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790257629041645-13911-0/minicluster-data/ts-1/wal/wals/d668e00555be4e298a5ad1a09670744e
I20260924 13:47:30.248594 18547 ts_tablet_manager.cc:1939] T d668e00555be4e298a5ad1a09670744e P 0b878d07b2d74f658d6f60629a7297b6: Deleting consensus metadata
I20260924 13:47:30.249135 18546 ts_tablet_manager.cc:1918] T d668e00555be4e298a5ad1a09670744e P e8be7acb295b4500844d25609fb0bb8d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 13:47:30.249120 18029 catalog_manager.cc:4841] TS 0b878d07b2d74f658d6f60629a7297b6 (127.13.149.194:33647): tablet d668e00555be4e298a5ad1a09670744e (table test-table [id=c823429b2e214ba185df62c7db28adf3]) successfully deleted
I20260924 13:47:30.249198 18546 log.cc:1196] T d668e00555be4e298a5ad1a09670744e P e8be7acb295b4500844d25609fb0bb8d: Deleting WAL directory at /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790257629041645-13911-0/minicluster-data/ts-2/wal/wals/d668e00555be4e298a5ad1a09670744e
I20260924 13:47:30.249449 18546 ts_tablet_manager.cc:1939] T d668e00555be4e298a5ad1a09670744e P e8be7acb295b4500844d25609fb0bb8d: Deleting consensus metadata
I20260924 13:47:30.249922 18031 catalog_manager.cc:4841] TS e8be7acb295b4500844d25609fb0bb8d (127.13.149.195:34105): tablet d668e00555be4e298a5ad1a09670744e (table test-table [id=c823429b2e214ba185df62c7db28adf3]) successfully deleted
I20260924 13:47:30.252697 13911 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJN9WRF/build/release/bin/kudu with pid 18242
W20260924 13:47:30.252748 18032 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.13.149.193:38301 (error 108)
I20260924 13:47:30.257184 13911 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJN9WRF/build/release/bin/kudu with pid 18376
I20260924 13:47:30.257779 18521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:30.257278 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.193:38301 (local address 127.0.0.1:54770)
0924 13:47:30.257684 (+ 406us) negotiation.cc:107] Waiting for socket to connect
0924 13:47:30.257734 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.149.193:38301: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":383}
W20260924 13:47:30.259052 18032 catalog_manager.cc:4584] TS aff50161196a4d46ad09d8617159bf3b (127.13.149.193:38301): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet d668e00555be4e298a5ad1a09670744e: Network error: Client connection negotiation failed: client connection to 127.13.149.193:38301: connect: Connection refused (error 111)
I20260924 13:47:30.261337 13911 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJN9WRF/build/release/bin/kudu with pid 18014
2026-09-24T13:47:30Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (1884 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-09-24T13:47:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T13:47:30Z Disabled control of system clock
I20260924 13:47:30.287827 13911 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJN9WRF/build/release/bin/kudu
/tmp/dist-test-taskJN9WRF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790257629041645-13911-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790257629041645-13911-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790257629041645-13911-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790257629041645-13911-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.149.254:34683
--webserver_interface=127.13.149.254
--webserver_port=0
--builtin_ntp_servers=127.13.149.212:35141
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.149.254:34683
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260924 13:47:30.372505 18553 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 13:47:30.372732 18553 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 13:47:30.372756 18553 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 13:47:30.374051 18553 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 13:47:30.374114 18553 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 13:47:30.374125 18553 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 13:47:30.374136 18553 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 13:47:30.374146 18553 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 13:47:30.375665 18553 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.149.212:35141
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790257629041645-13911-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790257629041645-13911-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.149.254:34683
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.149.254:34683
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790257629041645-13911-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.149.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18553
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790257629041645-13911-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 12:29:37 UTC on ccycloud.quasar-iqzbxn.root.comops.site
build id 4670106
I20260924 13:47:30.375957 18553 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 13:47:30.376272 18553 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 13:47:30.381165 18558 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 13:47:30.381232 18553 server_base.cc:1034] running on GCE node
W20260924 13:47:30.381184 18559 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 13:47:30.381184 18561 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 13:47:30.381606 18553 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 13:47:30.381961 18553 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 13:47:30.383108 18553 hybrid_clock.cc:648] HybridClock initialized: now 1790257650383093 us; error 37 us; skew 500 ppm
I20260924 13:47:30.384383 18553 webserver.cc:460] Webserver started at http://127.13.149.254:35195/ using document root <none> and password file <none>
I20260924 13:47:30.384601 18553 fs_manager.cc:359] Metadata directory not provided
I20260924 13:47:30.384650 18553 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 13:47:30.384776 18553 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 13:47:30.385928 18553 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790257629041645-13911-0/minicluster-data/master-0/data/instance:
uuid: "2067f395baab4c38bb51745a438d5c61"
format_stamp: "Formatted at 2026-09-24 13:47:30 on dist-test-slave-sjl2"
server_key: "fec186f86513c2388374ee87c82949fc"
server_key_iv: "e979959eea1dda3552ee509d2ad11250"
server_key_version: "encryptionkey@0"
I20260924 13:47:30.386278 18553 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790257629041645-13911-0/minicluster-data/master-0/wal/instance:
uuid: "2067f395baab4c38bb51745a438d5c61"
format_stamp: "Formatted at 2026-09-24 13:47:30 on dist-test-slave-sjl2"
server_key: "fec186f86513c2388374ee87c82949fc"
server_key_iv: "e979959eea1dda3552ee509d2ad11250"
server_key_version: "encryptionkey@0"
I20260924 13:47:30.387535 18553 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260924 13:47:30.388247 18567 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 13:47:30.388405 18553 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 13:47:30.388489 18553 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790257629041645-13911-0/minicluster-data/master-0/data,/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790257629041645-13911-0/minicluster-data/master-0/wal
uuid: "2067f395baab4c38bb51745a438d5c61"
format_stamp: "Formatted at 2026-09-24 13:47:30 on dist-test-slave-sjl2"
server_key: "fec186f86513c2388374ee87c82949fc"
server_key_iv: "e979959eea1dda3552ee509d2ad11250"
server_key_version: "encryptionkey@0"
I20260924 13:47:30.388556 18553 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790257629041645-13911-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790257629041645-13911-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790257629041645-13911-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 13:47:30.392011 18553 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 13:47:30.392292 18553 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 13:47:30.392414 18553 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 13:47:30.396275 18553 rpc_server.cc:307] RPC server started. Bound to: 127.13.149.254:34683
I20260924 13:47:30.396303 18620 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.149.254:34683 every 8 connection(s)
I20260924 13:47:30.396641 18553 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790257629041645-13911-0/minicluster-data/master-0/data/info.pb
I20260924 13:47:30.397362 18621 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 13:47:30.400293 18621 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2067f395baab4c38bb51745a438d5c61: Bootstrap starting.
I20260924 13:47:30.400957 18621 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2067f395baab4c38bb51745a438d5c61: Neither blocks nor log segments found. Creating new log.
I20260924 13:47:30.401263 18621 log.cc:824] T 00000000000000000000000000000000 P 2067f395baab4c38bb51745a438d5c61: Log is configured to *not* fsync() on all Append() calls
I20260924 13:47:30.402036 18621 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2067f395baab4c38bb51745a438d5c61: No bootstrap required, opened a new log
I20260924 13:47:30.402798 13911 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJN9WRF/build/release/bin/kudu as pid 18553
I20260924 13:47:30.402889 13911 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790257629041645-13911-0/minicluster-data/master-0/wal/instance
I20260924 13:47:30.403036 13911 external_mini_cluster.cc:1428] Setting key d4ebacd24f39e812a95ec4ade20363d6
I20260924 13:47:30.404182 18621 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2067f395baab4c38bb51745a438d5c61 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2067f395baab4c38bb51745a438d5c61" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 34683 } }
I20260924 13:47:30.404367 18621 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2067f395baab4c38bb51745a438d5c61 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 13:47:30.404405 18621 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2067f395baab4c38bb51745a438d5c61 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2067f395baab4c38bb51745a438d5c61, State: Initialized, Role: FOLLOWER
I20260924 13:47:30.404873 18621 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2067f395baab4c38bb51745a438d5c61 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2067f395baab4c38bb51745a438d5c61" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 34683 } }
I20260924 13:47:30.405023 18621 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2067f395baab4c38bb51745a438d5c61 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 13:47:30.405061 18621 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2067f395baab4c38bb51745a438d5c61 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 13:47:30.405112 18621 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2067f395baab4c38bb51745a438d5c61 [term 0 FOLLOWER]: Advancing to term 1
I20260924 13:47:30.405483 18625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:30.403583 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44016 (local address 127.13.149.254:34683)
0924 13:47:30.403836 (+ 253us) server_negotiation.cc:206] Beginning negotiation
0924 13:47:30.403846 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0924 13:47:30.403884 (+ 38us) server_negotiation.cc:407] Connection header received
0924 13:47:30.404134 (+ 250us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:47:30.404142 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:47:30.404193 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:47:30.404341 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:47:30.405014 (+ 673us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:47:30.405018 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:47:30.405018 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 13:47:30.405024 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:47:30.405073 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 13:47:30.405090 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:47:30.405163 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0924 13:47:30.405229 (+ 66us) server_negotiation.cc:299] Negotiation successful
0924 13:47:30.405291 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":159,"thread_start_us":104,"threads_started":1}
I20260924 13:47:30.406281 18621 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2067f395baab4c38bb51745a438d5c61 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2067f395baab4c38bb51745a438d5c61" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 34683 } }
I20260924 13:47:30.406375 18621 leader_election.cc:302] T 00000000000000000000000000000000 P 2067f395baab4c38bb51745a438d5c61 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2067f395baab4c38bb51745a438d5c61; no voters:
I20260924 13:47:30.406505 18621 leader_election.cc:288] T 00000000000000000000000000000000 P 2067f395baab4c38bb51745a438d5c61 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 13:47:30.406558 18626 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2067f395baab4c38bb51745a438d5c61 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 13:47:30.406731 18621 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2067f395baab4c38bb51745a438d5c61 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 13:47:30.406775 18626 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2067f395baab4c38bb51745a438d5c61 [term 1 LEADER]: Becoming Leader. State: Replica: 2067f395baab4c38bb51745a438d5c61, State: Running, Role: LEADER
I20260924 13:47:30.406891 18626 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2067f395baab4c38bb51745a438d5c61 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2067f395baab4c38bb51745a438d5c61" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 34683 } }
I20260924 13:47:30.407253 18627 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2067f395baab4c38bb51745a438d5c61 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2067f395baab4c38bb51745a438d5c61" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2067f395baab4c38bb51745a438d5c61" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 34683 } } }
I20260924 13:47:30.407369 18627 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2067f395baab4c38bb51745a438d5c61 [sys.catalog]: This master's current role is: LEADER
I20260924 13:47:30.407552 18635 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 13:47:30.407248 18628 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2067f395baab4c38bb51745a438d5c61 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2067f395baab4c38bb51745a438d5c61. Latest consensus state: current_term: 1 leader_uuid: "2067f395baab4c38bb51745a438d5c61" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2067f395baab4c38bb51745a438d5c61" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 34683 } } }
I20260924 13:47:30.407886 18628 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2067f395baab4c38bb51745a438d5c61 [sys.catalog]: This master's current role is: LEADER
I20260924 13:47:30.408115 18635 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 13:47:30.409924 18635 catalog_manager.cc:1347] Generated new cluster ID: 04caa555288a46e1b4c43fde408190d0
I20260924 13:47:30.409983 18635 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 13:47:30.435185 18635 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 13:47:30.435411 18635 catalog_manager.cc:1504] Loading token signing keys...
I20260924 13:47:30.443647 18635 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2067f395baab4c38bb51745a438d5c61: Generated new TSK 0
I20260924 13:47:30.443897 18635 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260924 13:47:30.450603 13911 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJN9WRF/build/release/bin/kudu
/tmp/dist-test-taskJN9WRF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790257629041645-13911-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790257629041645-13911-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790257629041645-13911-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790257629041645-13911-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.149.193:0
--local_ip_for_outbound_sockets=127.13.149.193
--webserver_interface=127.13.149.193
--webserver_port=0
--tserver_master_addrs=127.13.149.254:34683
--builtin_ntp_servers=127.13.149.212:35141
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260924 13:47:30.535137 18645 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 13:47:30.535357 18645 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 13:47:30.535391 18645 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 13:47:30.536641 18645 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 13:47:30.536703 18645 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 13:47:30.536744 18645 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.149.193
I20260924 13:47:30.538188 18645 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.149.212:35141
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790257629041645-13911-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790257629041645-13911-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.149.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790257629041645-13911-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.149.193
--webserver_port=0
--tserver_master_addrs=127.13.149.254:34683
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18645
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.149.193
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790257629041645-13911-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 12:29:37 UTC on ccycloud.quasar-iqzbxn.root.comops.site
build id 4670106
I20260924 13:47:30.538446 18645 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 13:47:30.538746 18645 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 13:47:30.543751 18650 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 13:47:30.543767 18651 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 13:47:30.543898 18645 server_base.cc:1034] running on GCE node
W20260924 13:47:30.543756 18653 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 13:47:30.544360 18645 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 13:47:30.544615 18645 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 13:47:30.545782 18645 hybrid_clock.cc:648] HybridClock initialized: now 1790257650545773 us; error 54 us; skew 500 ppm
I20260924 13:47:30.552386 18645 webserver.cc:460] Webserver started at http://127.13.149.193:46189/ using document root <none> and password file <none>
I20260924 13:47:30.552717 18645 fs_manager.cc:359] Metadata directory not provided
I20260924 13:47:30.552784 18645 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 13:47:30.552947 18645 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 13:47:30.554090 18645 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790257629041645-13911-0/minicluster-data/ts-0/data/instance:
uuid: "f4854a67282b4b43aa5e59ea38c69144"
format_stamp: "Formatted at 2026-09-24 13:47:30 on dist-test-slave-sjl2"
server_key: "1e0bad95a3238fd30a7acbdd90028c3a"
server_key_iv: "cfef244bb03b6ddd33e8db154e3661fa"
server_key_version: "encryptionkey@0"
I20260924 13:47:30.554436 18645 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790257629041645-13911-0/minicluster-data/ts-0/wal/instance:
uuid: "f4854a67282b4b43aa5e59ea38c69144"
format_stamp: "Formatted at 2026-09-24 13:47:30 on dist-test-slave-sjl2"
server_key: "1e0bad95a3238fd30a7acbdd90028c3a"
server_key_iv: "cfef244bb03b6ddd33e8db154e3661fa"
server_key_version: "encryptionkey@0"
I20260924 13:47:30.555802 18645 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260924 13:47:30.556527 18659 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 13:47:30.556710 18645 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 13:47:30.556797 18645 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790257629041645-13911-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790257629041645-13911-0/minicluster-data/ts-0/wal
uuid: "f4854a67282b4b43aa5e59ea38c69144"
format_stamp: "Formatted at 2026-09-24 13:47:30 on dist-test-slave-sjl2"
server_key: "1e0bad95a3238fd30a7acbdd90028c3a"
server_key_iv: "cfef244bb03b6ddd33e8db154e3661fa"
server_key_version: "encryptionkey@0"
I20260924 13:47:30.556866 18645 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790257629041645-13911-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790257629041645-13911-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790257629041645-13911-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 13:47:30.560355 18645 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 13:47:30.560647 18645 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 13:47:30.560773 18645 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 13:47:30.560997 18645 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 13:47:30.561313 18645 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 13:47:30.561348 18645 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 13:47:30.561376 18645 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 13:47:30.561401 18645 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 13:47:30.567011 18645 rpc_server.cc:307] RPC server started. Bound to: 127.13.149.193:40315
I20260924 13:47:30.567080 18772 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.149.193:40315 every 8 connection(s)
I20260924 13:47:30.567389 18645 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790257629041645-13911-0/minicluster-data/ts-0/data/info.pb
I20260924 13:47:30.570559 18625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:30.568107 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.149.193:44951 (local address 127.13.149.254:34683)
0924 13:47:30.568171 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0924 13:47:30.568174 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 13:47:30.568432 (+ 258us) server_negotiation.cc:407] Connection header received
0924 13:47:30.568997 (+ 565us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:47:30.568998 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:47:30.569004 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:47:30.569065 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:47:30.570253 (+ 1188us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:47:30.570255 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:47:30.570255 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 13:47:30.570259 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:47:30.570272 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 13:47:30.570276 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:47:30.570324 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0924 13:47:30.570480 (+ 156us) server_negotiation.cc:299] Negotiation successful
0924 13:47:30.570498 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260924 13:47:30.570698 18775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:30.568099 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:34683 (local address 127.13.149.193:44951)
0924 13:47:30.568314 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0924 13:47:30.568352 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0924 13:47:30.568871 (+ 519us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:47:30.569142 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:47:30.569145 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:47:30.569155 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0924 13:47:30.570090 (+ 935us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0924 13:47:30.570090 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0924 13:47:30.570122 (+ 32us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0924 13:47:30.570163 (+ 41us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 13:47:30.570360 (+ 197us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 13:47:30.570367 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 13:47:30.570370 (+ 3us) client_negotiation.cc:769] Sending connection context
0924 13:47:30.570453 (+ 83us) client_negotiation.cc:240] Negotiation successful
0924 13:47:30.570512 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":96,"threads_started":1}
I20260924 13:47:30.571657 18773 heartbeater.cc:347] Connected to a master server at 127.13.149.254:34683
I20260924 13:47:30.571776 18773 heartbeater.cc:464] Registering TS with master...
I20260924 13:47:30.571944 18773 heartbeater.cc:511] Master 127.13.149.254:34683 requested a full tablet report, sending...
I20260924 13:47:30.572549 18584 ts_manager.cc:194] Registered new tserver with Master: f4854a67282b4b43aa5e59ea38c69144 (127.13.149.193:40315)
I20260924 13:47:30.575834 13911 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJN9WRF/build/release/bin/kudu as pid 18645
I20260924 13:47:30.575942 13911 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790257629041645-13911-0/minicluster-data/ts-0/wal/instance
I20260924 13:47:30.576104 13911 external_mini_cluster.cc:1428] Setting key 342187bf8909a5f92050e1f7ba28a610
I20260924 13:47:30.577121 13911 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJN9WRF/build/release/bin/kudu
/tmp/dist-test-taskJN9WRF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790257629041645-13911-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790257629041645-13911-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790257629041645-13911-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790257629041645-13911-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.149.194:0
--local_ip_for_outbound_sockets=127.13.149.194
--webserver_interface=127.13.149.194
--webserver_port=0
--tserver_master_addrs=127.13.149.254:34683
--builtin_ntp_servers=127.13.149.212:35141
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260924 13:47:30.661703 18776 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 13:47:30.661937 18776 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 13:47:30.661962 18776 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 13:47:30.663329 18776 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 13:47:30.663388 18776 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 13:47:30.663417 18776 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.149.194
I20260924 13:47:30.664904 18776 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.149.212:35141
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790257629041645-13911-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790257629041645-13911-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.149.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790257629041645-13911-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.149.194
--webserver_port=0
--tserver_master_addrs=127.13.149.254:34683
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18776
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.149.194
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790257629041645-13911-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 12:29:37 UTC on ccycloud.quasar-iqzbxn.root.comops.site
build id 4670106
I20260924 13:47:30.665191 18776 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 13:47:30.665491 18776 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 13:47:30.670668 18782 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 13:47:30.670677 18785 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 13:47:30.670662 18781 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 13:47:30.670711 18776 server_base.cc:1034] running on GCE node
I20260924 13:47:30.671181 18776 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 13:47:30.671434 18776 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 13:47:30.672578 18776 hybrid_clock.cc:648] HybridClock initialized: now 1790257650672572 us; error 37 us; skew 500 ppm
I20260924 13:47:30.677423 18776 webserver.cc:460] Webserver started at http://127.13.149.194:39731/ using document root <none> and password file <none>
I20260924 13:47:30.677677 18776 fs_manager.cc:359] Metadata directory not provided
I20260924 13:47:30.677732 18776 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 13:47:30.677910 18776 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 13:47:30.679010 18776 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790257629041645-13911-0/minicluster-data/ts-1/data/instance:
uuid: "80d29b3ad60748d6be19519e23c320c7"
format_stamp: "Formatted at 2026-09-24 13:47:30 on dist-test-slave-sjl2"
server_key: "be58505e75fc4a5763b441417cd7687b"
server_key_iv: "887af21e9bcdd82c1d36e5ed47f970dd"
server_key_version: "encryptionkey@0"
I20260924 13:47:30.679363 18776 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790257629041645-13911-0/minicluster-data/ts-1/wal/instance:
uuid: "80d29b3ad60748d6be19519e23c320c7"
format_stamp: "Formatted at 2026-09-24 13:47:30 on dist-test-slave-sjl2"
server_key: "be58505e75fc4a5763b441417cd7687b"
server_key_iv: "887af21e9bcdd82c1d36e5ed47f970dd"
server_key_version: "encryptionkey@0"
I20260924 13:47:30.680613 18776 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260924 13:47:30.681414 18791 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 13:47:30.681565 18776 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260924 13:47:30.681648 18776 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790257629041645-13911-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790257629041645-13911-0/minicluster-data/ts-1/wal
uuid: "80d29b3ad60748d6be19519e23c320c7"
format_stamp: "Formatted at 2026-09-24 13:47:30 on dist-test-slave-sjl2"
server_key: "be58505e75fc4a5763b441417cd7687b"
server_key_iv: "887af21e9bcdd82c1d36e5ed47f970dd"
server_key_version: "encryptionkey@0"
I20260924 13:47:30.681715 18776 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790257629041645-13911-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790257629041645-13911-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790257629041645-13911-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 13:47:30.685222 18776 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 13:47:30.685497 18776 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 13:47:30.685632 18776 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 13:47:30.685844 18776 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 13:47:30.686136 18776 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 13:47:30.686168 18776 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 13:47:30.686197 18776 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 13:47:30.686220 18776 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 13:47:30.691956 18776 rpc_server.cc:307] RPC server started. Bound to: 127.13.149.194:41447
I20260924 13:47:30.692020 18906 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.149.194:41447 every 8 connection(s)
I20260924 13:47:30.692366 18776 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790257629041645-13911-0/minicluster-data/ts-1/data/info.pb
I20260924 13:47:30.695293 18625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:30.693083 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.149.194:34087 (local address 127.13.149.254:34683)
0924 13:47:30.693130 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0924 13:47:30.693132 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:47:30.693428 (+ 296us) server_negotiation.cc:407] Connection header received
0924 13:47:30.693930 (+ 502us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:47:30.693931 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:47:30.693936 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:47:30.693989 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:47:30.695114 (+ 1125us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:47:30.695115 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:47:30.695115 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 13:47:30.695117 (+ 2us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:47:30.695130 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 13:47:30.695134 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:47:30.695170 (+ 36us) server_negotiation.cc:1035] Waiting for connection context
0924 13:47:30.695224 (+ 54us) server_negotiation.cc:299] Negotiation successful
0924 13:47:30.695236 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260924 13:47:30.695482 18909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:30.693111 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:34683 (local address 127.13.149.194:34087)
0924 13:47:30.693301 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0924 13:47:30.693338 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0924 13:47:30.693834 (+ 496us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:47:30.694047 (+ 213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:47:30.694050 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:47:30.694059 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0924 13:47:30.694982 (+ 923us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0924 13:47:30.694983 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0924 13:47:30.695014 (+ 31us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0924 13:47:30.695045 (+ 31us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 13:47:30.695173 (+ 128us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 13:47:30.695176 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 13:47:30.695178 (+ 2us) client_negotiation.cc:769] Sending connection context
0924 13:47:30.695228 (+ 50us) client_negotiation.cc:240] Negotiation successful
0924 13:47:30.695285 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":76,"threads_started":1}
I20260924 13:47:30.695909 18907 heartbeater.cc:347] Connected to a master server at 127.13.149.254:34683
I20260924 13:47:30.695999 18907 heartbeater.cc:464] Registering TS with master...
I20260924 13:47:30.696118 18907 heartbeater.cc:511] Master 127.13.149.254:34683 requested a full tablet report, sending...
I20260924 13:47:30.696435 18584 ts_manager.cc:194] Registered new tserver with Master: 80d29b3ad60748d6be19519e23c320c7 (127.13.149.194:41447)
I20260924 13:47:30.702064 13911 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJN9WRF/build/release/bin/kudu as pid 18776
I20260924 13:47:30.702155 13911 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790257629041645-13911-0/minicluster-data/ts-1/wal/instance
I20260924 13:47:30.702299 13911 external_mini_cluster.cc:1428] Setting key 94727a745fd6607d499e6b6b56fd4251
I20260924 13:47:30.703413 13911 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJN9WRF/build/release/bin/kudu
/tmp/dist-test-taskJN9WRF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790257629041645-13911-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790257629041645-13911-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790257629041645-13911-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790257629041645-13911-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.149.195:0
--local_ip_for_outbound_sockets=127.13.149.195
--webserver_interface=127.13.149.195
--webserver_port=0
--tserver_master_addrs=127.13.149.254:34683
--builtin_ntp_servers=127.13.149.212:35141
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260924 13:47:30.796976 18910 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 13:47:30.797291 18910 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 13:47:30.797326 18910 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 13:47:30.799243 18910 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 13:47:30.799322 18910 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 13:47:30.799368 18910 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.149.195
I20260924 13:47:30.801041 18910 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.149.212:35141
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790257629041645-13911-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790257629041645-13911-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.149.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790257629041645-13911-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.149.195
--webserver_port=0
--tserver_master_addrs=127.13.149.254:34683
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18910
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.149.195
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790257629041645-13911-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 12:29:37 UTC on ccycloud.quasar-iqzbxn.root.comops.site
build id 4670106
I20260924 13:47:30.801330 18910 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 13:47:30.801651 18910 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 13:47:30.806686 18915 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 13:47:30.806663 18916 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 13:47:30.806710 18918 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 13:47:30.806808 18910 server_base.cc:1034] running on GCE node
I20260924 13:47:30.807169 18910 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 13:47:30.807456 18910 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 13:47:30.808640 18910 hybrid_clock.cc:648] HybridClock initialized: now 1790257650808606 us; error 39 us; skew 500 ppm
I20260924 13:47:30.810020 18910 webserver.cc:460] Webserver started at http://127.13.149.195:34485/ using document root <none> and password file <none>
I20260924 13:47:30.810272 18910 fs_manager.cc:359] Metadata directory not provided
I20260924 13:47:30.810328 18910 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 13:47:30.810460 18910 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 13:47:30.811609 18910 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790257629041645-13911-0/minicluster-data/ts-2/data/instance:
uuid: "71dd31eab16645cd8e0104029a46c6f5"
format_stamp: "Formatted at 2026-09-24 13:47:30 on dist-test-slave-sjl2"
server_key: "6711a0fb4c8c7778c2a8cc9c4b8b89b6"
server_key_iv: "a5f22612406f75f630dba5cae526f43f"
server_key_version: "encryptionkey@0"
I20260924 13:47:30.811959 18910 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790257629041645-13911-0/minicluster-data/ts-2/wal/instance:
uuid: "71dd31eab16645cd8e0104029a46c6f5"
format_stamp: "Formatted at 2026-09-24 13:47:30 on dist-test-slave-sjl2"
server_key: "6711a0fb4c8c7778c2a8cc9c4b8b89b6"
server_key_iv: "a5f22612406f75f630dba5cae526f43f"
server_key_version: "encryptionkey@0"
I20260924 13:47:30.813401 18910 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260924 13:47:30.814168 18924 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 13:47:30.814332 18910 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 13:47:30.814420 18910 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790257629041645-13911-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790257629041645-13911-0/minicluster-data/ts-2/wal
uuid: "71dd31eab16645cd8e0104029a46c6f5"
format_stamp: "Formatted at 2026-09-24 13:47:30 on dist-test-slave-sjl2"
server_key: "6711a0fb4c8c7778c2a8cc9c4b8b89b6"
server_key_iv: "a5f22612406f75f630dba5cae526f43f"
server_key_version: "encryptionkey@0"
I20260924 13:47:30.814495 18910 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790257629041645-13911-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790257629041645-13911-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790257629041645-13911-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 13:47:30.818037 18910 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 13:47:30.818316 18910 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 13:47:30.818441 18910 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 13:47:30.818634 18910 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 13:47:30.818936 18910 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 13:47:30.818969 18910 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 13:47:30.818995 18910 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 13:47:30.819018 18910 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 13:47:30.824503 18910 rpc_server.cc:307] RPC server started. Bound to: 127.13.149.195:37641
I20260924 13:47:30.824551 19038 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.149.195:37641 every 8 connection(s)
I20260924 13:47:30.824883 18910 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790257629041645-13911-0/minicluster-data/ts-2/data/info.pb
I20260924 13:47:30.827818 18625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:30.825633 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.149.195:58961 (local address 127.13.149.254:34683)
0924 13:47:30.825683 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0924 13:47:30.825685 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:47:30.825886 (+ 201us) server_negotiation.cc:407] Connection header received
0924 13:47:30.826347 (+ 461us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:47:30.826348 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:47:30.826356 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:47:30.826408 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:47:30.827593 (+ 1185us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:47:30.827594 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:47:30.827594 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 13:47:30.827599 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:47:30.827614 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 13:47:30.827619 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:47:30.827685 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0924 13:47:30.827742 (+ 57us) server_negotiation.cc:299] Negotiation successful
0924 13:47:30.827758 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260924 13:47:30.827996 19041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:30.825605 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:34683 (local address 127.13.149.195:58961)
0924 13:47:30.825776 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0924 13:47:30.825808 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0924 13:47:30.826253 (+ 445us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:47:30.826460 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:47:30.826463 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:47:30.826472 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0924 13:47:30.827402 (+ 930us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0924 13:47:30.827402 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0924 13:47:30.827436 (+ 34us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0924 13:47:30.827466 (+ 30us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 13:47:30.827685 (+ 219us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 13:47:30.827688 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 13:47:30.827691 (+ 3us) client_negotiation.cc:769] Sending connection context
0924 13:47:30.827741 (+ 50us) client_negotiation.cc:240] Negotiation successful
0924 13:47:30.827796 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":75,"threads_started":1}
I20260924 13:47:30.828111 13911 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJN9WRF/build/release/bin/kudu as pid 18910
I20260924 13:47:30.828218 13911 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790257629041645-13911-0/minicluster-data/ts-2/wal/instance
I20260924 13:47:30.828401 13911 external_mini_cluster.cc:1428] Setting key 4d3b8ad166a65d52e882e6b661a1a39c
I20260924 13:47:30.828500 19039 heartbeater.cc:347] Connected to a master server at 127.13.149.254:34683
I20260924 13:47:30.828604 19039 heartbeater.cc:464] Registering TS with master...
I20260924 13:47:30.828763 19039 heartbeater.cc:511] Master 127.13.149.254:34683 requested a full tablet report, sending...
I20260924 13:47:30.829154 18583 ts_manager.cc:194] Registered new tserver with Master: 71dd31eab16645cd8e0104029a46c6f5 (127.13.149.195:37641)
I20260924 13:47:30.840126 13911 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260924 13:47:30.842479 18625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:30.841510 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44020 (local address 127.13.149.254:34683)
0924 13:47:30.841579 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0924 13:47:30.841582 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 13:47:30.841642 (+ 60us) server_negotiation.cc:407] Connection header received
0924 13:47:30.841701 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:47:30.841702 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:47:30.841706 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:47:30.841753 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:47:30.842298 (+ 545us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:47:30.842299 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:47:30.842299 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 13:47:30.842301 (+ 2us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:47:30.842310 (+ 9us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 13:47:30.842314 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:47:30.842348 (+ 34us) server_negotiation.cc:1035] Waiting for connection context
0924 13:47:30.842380 (+ 32us) server_negotiation.cc:299] Negotiation successful
0924 13:47:30.842390 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260924 13:47:30.844147 18583 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44020:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 13:47:30.845883 18583 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 13:47:30.849419 19047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:30.847953 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.193:40315 (local address 127.0.0.1:48376)
0924 13:47:30.848208 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0924 13:47:30.848217 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0924 13:47:30.848269 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:47:30.848399 (+ 130us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:47:30.848405 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:47:30.848412 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0924 13:47:30.848888 (+ 476us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0924 13:47:30.848889 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0924 13:47:30.848907 (+ 18us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0924 13:47:30.848959 (+ 52us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 13:47:30.849220 (+ 261us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 13:47:30.849227 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 13:47:30.849230 (+ 3us) client_negotiation.cc:769] Sending connection context
0924 13:47:30.849317 (+ 87us) client_negotiation.cc:240] Negotiation successful
0924 13:47:30.849337 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":240,"thread_start_us":126,"threads_started":1}
I20260924 13:47:30.849859 19049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:30.848049 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48376 (local address 127.13.149.193:40315)
0924 13:47:30.848192 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0924 13:47:30.848196 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 13:47:30.848269 (+ 73us) server_negotiation.cc:407] Connection header received
0924 13:47:30.848326 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:47:30.848329 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:47:30.848338 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:47:30.848400 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:47:30.849034 (+ 634us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:47:30.849045 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:47:30.849046 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 13:47:30.849051 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:47:30.849093 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 13:47:30.849126 (+ 33us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:47:30.849191 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0924 13:47:30.849758 (+ 567us) server_negotiation.cc:299] Negotiation successful
0924 13:47:30.849777 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":106,"thread_start_us":78,"threads_started":1}
I20260924 13:47:30.849972 19051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:30.848344 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56142 (local address 127.13.149.195:37641)
0924 13:47:30.848500 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0924 13:47:30.848504 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 13:47:30.848515 (+ 11us) server_negotiation.cc:407] Connection header received
0924 13:47:30.848543 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:47:30.848545 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:47:30.848555 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:47:30.848955 (+ 400us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:47:30.849535 (+ 580us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:47:30.849539 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:47:30.849539 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 13:47:30.849543 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:47:30.849573 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 13:47:30.849588 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:47:30.849851 (+ 263us) server_negotiation.cc:1035] Waiting for connection context
0924 13:47:30.849876 (+ 25us) server_negotiation.cc:299] Negotiation successful
0924 13:47:30.849892 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":119,"thread_start_us":73,"threads_started":1}
I20260924 13:47:30.850181 19050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:30.848193 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.194:41447 (local address 127.0.0.1:42084)
0924 13:47:30.848840 (+ 647us) negotiation.cc:107] Waiting for socket to connect
0924 13:47:30.848851 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0924 13:47:30.848895 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:47:30.849438 (+ 543us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:47:30.849439 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:47:30.849441 (+ 2us) client_negotiation.cc:189] Negotiated authn=SASL
0924 13:47:30.849779 (+ 338us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0924 13:47:30.849779 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0924 13:47:30.849786 (+ 7us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0924 13:47:30.849797 (+ 11us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 13:47:30.850066 (+ 269us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 13:47:30.850067 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 13:47:30.850068 (+ 1us) client_negotiation.cc:769] Sending connection context
0924 13:47:30.850107 (+ 39us) client_negotiation.cc:240] Negotiation successful
0924 13:47:30.850119 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":621,"mutex_wait_us":2,"thread_start_us":74,"threads_started":1}
I20260924 13:47:30.849782 19048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:30.847950 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.195:37641 (local address 127.0.0.1:56142)
0924 13:47:30.848154 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0924 13:47:30.848197 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0924 13:47:30.848257 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:47:30.848667 (+ 410us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:47:30.848668 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:47:30.848671 (+ 3us) client_negotiation.cc:189] Negotiated authn=SASL
0924 13:47:30.849160 (+ 489us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0924 13:47:30.849161 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0924 13:47:30.849168 (+ 7us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0924 13:47:30.849180 (+ 12us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 13:47:30.849672 (+ 492us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 13:47:30.849673 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 13:47:30.849674 (+ 1us) client_negotiation.cc:769] Sending connection context
0924 13:47:30.849705 (+ 31us) client_negotiation.cc:240] Negotiation successful
0924 13:47:30.849719 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":73,"threads_started":1}
I20260924 13:47:30.850368 19052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:30.848639 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42084 (local address 127.13.149.194:41447)
0924 13:47:30.849052 (+ 413us) server_negotiation.cc:206] Beginning negotiation
0924 13:47:30.849056 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 13:47:30.849066 (+ 10us) server_negotiation.cc:407] Connection header received
0924 13:47:30.849090 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:47:30.849093 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:47:30.849101 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:47:30.849170 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:47:30.849870 (+ 700us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:47:30.849877 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:47:30.849878 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 13:47:30.849882 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:47:30.849915 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 13:47:30.849993 (+ 78us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:47:30.850042 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0924 13:47:30.850272 (+ 230us) server_negotiation.cc:299] Negotiation successful
0924 13:47:30.850289 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":159,"thread_start_us":99,"threads_started":1}
I20260924 13:47:30.850710 18707 tablet_service.cc:1467] Processing CreateTablet for tablet fb9e5c3b2c784ca5aa7d5865b7da4194 (DEFAULT_TABLE table=test-table [id=667e10491d5b4de9b33c3ce7b1fdab29]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 13:47:30.851052 18707 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fb9e5c3b2c784ca5aa7d5865b7da4194. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 13:47:30.850963 18840 tablet_service.cc:1467] Processing CreateTablet for tablet fb9e5c3b2c784ca5aa7d5865b7da4194 (DEFAULT_TABLE table=test-table [id=667e10491d5b4de9b33c3ce7b1fdab29]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 13:47:30.851315 18840 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fb9e5c3b2c784ca5aa7d5865b7da4194. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 13:47:30.853680 18972 tablet_service.cc:1467] Processing CreateTablet for tablet fb9e5c3b2c784ca5aa7d5865b7da4194 (DEFAULT_TABLE table=test-table [id=667e10491d5b4de9b33c3ce7b1fdab29]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 13:47:30.853883 19053 tablet_bootstrap.cc:492] T fb9e5c3b2c784ca5aa7d5865b7da4194 P f4854a67282b4b43aa5e59ea38c69144: Bootstrap starting.
I20260924 13:47:30.854034 18972 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fb9e5c3b2c784ca5aa7d5865b7da4194. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 13:47:30.854518 19053 tablet_bootstrap.cc:654] T fb9e5c3b2c784ca5aa7d5865b7da4194 P f4854a67282b4b43aa5e59ea38c69144: Neither blocks nor log segments found. Creating new log.
I20260924 13:47:30.854836 19053 log.cc:824] T fb9e5c3b2c784ca5aa7d5865b7da4194 P f4854a67282b4b43aa5e59ea38c69144: Log is configured to *not* fsync() on all Append() calls
I20260924 13:47:30.855177 19054 tablet_bootstrap.cc:492] T fb9e5c3b2c784ca5aa7d5865b7da4194 P 80d29b3ad60748d6be19519e23c320c7: Bootstrap starting.
I20260924 13:47:30.855561 19053 tablet_bootstrap.cc:492] T fb9e5c3b2c784ca5aa7d5865b7da4194 P f4854a67282b4b43aa5e59ea38c69144: No bootstrap required, opened a new log
I20260924 13:47:30.855628 19053 ts_tablet_manager.cc:1397] T fb9e5c3b2c784ca5aa7d5865b7da4194 P f4854a67282b4b43aa5e59ea38c69144: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260924 13:47:30.855831 19054 tablet_bootstrap.cc:654] T fb9e5c3b2c784ca5aa7d5865b7da4194 P 80d29b3ad60748d6be19519e23c320c7: Neither blocks nor log segments found. Creating new log.
I20260924 13:47:30.856128 19054 log.cc:824] T fb9e5c3b2c784ca5aa7d5865b7da4194 P 80d29b3ad60748d6be19519e23c320c7: Log is configured to *not* fsync() on all Append() calls
I20260924 13:47:30.856822 19054 tablet_bootstrap.cc:492] T fb9e5c3b2c784ca5aa7d5865b7da4194 P 80d29b3ad60748d6be19519e23c320c7: No bootstrap required, opened a new log
I20260924 13:47:30.856891 19054 ts_tablet_manager.cc:1397] T fb9e5c3b2c784ca5aa7d5865b7da4194 P 80d29b3ad60748d6be19519e23c320c7: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260924 13:47:30.857425 19057 tablet_bootstrap.cc:492] T fb9e5c3b2c784ca5aa7d5865b7da4194 P 71dd31eab16645cd8e0104029a46c6f5: Bootstrap starting.
I20260924 13:47:30.858143 19057 tablet_bootstrap.cc:654] T fb9e5c3b2c784ca5aa7d5865b7da4194 P 71dd31eab16645cd8e0104029a46c6f5: Neither blocks nor log segments found. Creating new log.
I20260924 13:47:30.858225 19053 raft_consensus.cc:348] T fb9e5c3b2c784ca5aa7d5865b7da4194 P f4854a67282b4b43aa5e59ea38c69144 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4854a67282b4b43aa5e59ea38c69144" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 40315 } } peers { permanent_uuid: "71dd31eab16645cd8e0104029a46c6f5" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 37641 } } peers { permanent_uuid: "80d29b3ad60748d6be19519e23c320c7" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 41447 } }
I20260924 13:47:30.858404 19053 raft_consensus.cc:374] T fb9e5c3b2c784ca5aa7d5865b7da4194 P f4854a67282b4b43aa5e59ea38c69144 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 13:47:30.858440 19053 raft_consensus.cc:729] T fb9e5c3b2c784ca5aa7d5865b7da4194 P f4854a67282b4b43aa5e59ea38c69144 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f4854a67282b4b43aa5e59ea38c69144, State: Initialized, Role: FOLLOWER
I20260924 13:47:30.858464 19057 log.cc:824] T fb9e5c3b2c784ca5aa7d5865b7da4194 P 71dd31eab16645cd8e0104029a46c6f5: Log is configured to *not* fsync() on all Append() calls
I20260924 13:47:30.858579 19053 consensus_queue.cc:260] T fb9e5c3b2c784ca5aa7d5865b7da4194 P f4854a67282b4b43aa5e59ea38c69144 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4854a67282b4b43aa5e59ea38c69144" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 40315 } } peers { permanent_uuid: "71dd31eab16645cd8e0104029a46c6f5" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 37641 } } peers { permanent_uuid: "80d29b3ad60748d6be19519e23c320c7" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 41447 } }
I20260924 13:47:30.858856 19053 ts_tablet_manager.cc:1428] T fb9e5c3b2c784ca5aa7d5865b7da4194 P f4854a67282b4b43aa5e59ea38c69144: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260924 13:47:30.858897 18773 heartbeater.cc:503] Master 127.13.149.254:34683 was elected leader, sending a full tablet report...
I20260924 13:47:30.858913 19054 raft_consensus.cc:348] T fb9e5c3b2c784ca5aa7d5865b7da4194 P 80d29b3ad60748d6be19519e23c320c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4854a67282b4b43aa5e59ea38c69144" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 40315 } } peers { permanent_uuid: "71dd31eab16645cd8e0104029a46c6f5" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 37641 } } peers { permanent_uuid: "80d29b3ad60748d6be19519e23c320c7" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 41447 } }
I20260924 13:47:30.859064 19054 raft_consensus.cc:374] T fb9e5c3b2c784ca5aa7d5865b7da4194 P 80d29b3ad60748d6be19519e23c320c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 13:47:30.859097 19054 raft_consensus.cc:729] T fb9e5c3b2c784ca5aa7d5865b7da4194 P 80d29b3ad60748d6be19519e23c320c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80d29b3ad60748d6be19519e23c320c7, State: Initialized, Role: FOLLOWER
I20260924 13:47:30.859207 19054 consensus_queue.cc:260] T fb9e5c3b2c784ca5aa7d5865b7da4194 P 80d29b3ad60748d6be19519e23c320c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4854a67282b4b43aa5e59ea38c69144" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 40315 } } peers { permanent_uuid: "71dd31eab16645cd8e0104029a46c6f5" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 37641 } } peers { permanent_uuid: "80d29b3ad60748d6be19519e23c320c7" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 41447 } }
I20260924 13:47:30.859459 19054 ts_tablet_manager.cc:1428] T fb9e5c3b2c784ca5aa7d5865b7da4194 P 80d29b3ad60748d6be19519e23c320c7: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260924 13:47:30.859494 18907 heartbeater.cc:503] Master 127.13.149.254:34683 was elected leader, sending a full tablet report...
I20260924 13:47:30.859551 19057 tablet_bootstrap.cc:492] T fb9e5c3b2c784ca5aa7d5865b7da4194 P 71dd31eab16645cd8e0104029a46c6f5: No bootstrap required, opened a new log
I20260924 13:47:30.859611 19057 ts_tablet_manager.cc:1397] T fb9e5c3b2c784ca5aa7d5865b7da4194 P 71dd31eab16645cd8e0104029a46c6f5: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260924 13:47:30.861727 19057 raft_consensus.cc:348] T fb9e5c3b2c784ca5aa7d5865b7da4194 P 71dd31eab16645cd8e0104029a46c6f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4854a67282b4b43aa5e59ea38c69144" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 40315 } } peers { permanent_uuid: "71dd31eab16645cd8e0104029a46c6f5" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 37641 } } peers { permanent_uuid: "80d29b3ad60748d6be19519e23c320c7" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 41447 } }
I20260924 13:47:30.861877 19057 raft_consensus.cc:374] T fb9e5c3b2c784ca5aa7d5865b7da4194 P 71dd31eab16645cd8e0104029a46c6f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 13:47:30.861909 19057 raft_consensus.cc:729] T fb9e5c3b2c784ca5aa7d5865b7da4194 P 71dd31eab16645cd8e0104029a46c6f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 71dd31eab16645cd8e0104029a46c6f5, State: Initialized, Role: FOLLOWER
I20260924 13:47:30.862018 19057 consensus_queue.cc:260] T fb9e5c3b2c784ca5aa7d5865b7da4194 P 71dd31eab16645cd8e0104029a46c6f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4854a67282b4b43aa5e59ea38c69144" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 40315 } } peers { permanent_uuid: "71dd31eab16645cd8e0104029a46c6f5" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 37641 } } peers { permanent_uuid: "80d29b3ad60748d6be19519e23c320c7" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 41447 } }
I20260924 13:47:30.862248 19057 ts_tablet_manager.cc:1428] T fb9e5c3b2c784ca5aa7d5865b7da4194 P 71dd31eab16645cd8e0104029a46c6f5: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260924 13:47:30.862288 19039 heartbeater.cc:503] Master 127.13.149.254:34683 was elected leader, sending a full tablet report...
W20260924 13:47:30.942896 18908 tablet.cc:2369] T fb9e5c3b2c784ca5aa7d5865b7da4194 P 80d29b3ad60748d6be19519e23c320c7: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260924 13:47:31.027334 19059 raft_consensus.cc:482] T fb9e5c3b2c784ca5aa7d5865b7da4194 P f4854a67282b4b43aa5e59ea38c69144 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 13:47:31.027498 19059 raft_consensus.cc:504] T fb9e5c3b2c784ca5aa7d5865b7da4194 P f4854a67282b4b43aa5e59ea38c69144 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4854a67282b4b43aa5e59ea38c69144" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 40315 } } peers { permanent_uuid: "71dd31eab16645cd8e0104029a46c6f5" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 37641 } } peers { permanent_uuid: "80d29b3ad60748d6be19519e23c320c7" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 41447 } }
I20260924 13:47:31.027876 19059 leader_election.cc:288] T fb9e5c3b2c784ca5aa7d5865b7da4194 P f4854a67282b4b43aa5e59ea38c69144 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 71dd31eab16645cd8e0104029a46c6f5 (127.13.149.195:37641), 80d29b3ad60748d6be19519e23c320c7 (127.13.149.194:41447)
I20260924 13:47:31.029320 18775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:31.028115 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.195:37641 (local address 127.13.149.193:51241)
0924 13:47:31.028176 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0924 13:47:31.028188 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0924 13:47:31.028231 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:47:31.028429 (+ 198us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:47:31.028429 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:47:31.028431 (+ 2us) client_negotiation.cc:189] Negotiated authn=SASL
0924 13:47:31.028953 (+ 522us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0924 13:47:31.028954 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0924 13:47:31.028964 (+ 10us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0924 13:47:31.028980 (+ 16us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 13:47:31.029191 (+ 211us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 13:47:31.029192 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 13:47:31.029193 (+ 1us) client_negotiation.cc:769] Sending connection context
0924 13:47:31.029232 (+ 39us) client_negotiation.cc:240] Negotiation successful
0924 13:47:31.029247 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43}
I20260924 13:47:31.029328 19051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:31.028252 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.149.193:51241 (local address 127.13.149.195:37641)
0924 13:47:31.028291 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0924 13:47:31.028293 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:47:31.028301 (+ 8us) server_negotiation.cc:407] Connection header received
0924 13:47:31.028329 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:47:31.028330 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:47:31.028334 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:47:31.028412 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:47:31.029092 (+ 680us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:47:31.029094 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:47:31.029095 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 13:47:31.029100 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:47:31.029121 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 13:47:31.029126 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:47:31.029181 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0924 13:47:31.029249 (+ 68us) server_negotiation.cc:299] Negotiation successful
0924 13:47:31.029263 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260924 13:47:31.029405 19052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:31.028220 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.149.193:35853 (local address 127.13.149.194:41447)
0924 13:47:31.028265 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0924 13:47:31.028267 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:47:31.028373 (+ 106us) server_negotiation.cc:407] Connection header received
0924 13:47:31.028400 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:47:31.028401 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:47:31.028405 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:47:31.028457 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:47:31.029173 (+ 716us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:47:31.029174 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:47:31.029175 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 13:47:31.029180 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:47:31.029199 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 13:47:31.029203 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:47:31.029250 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0924 13:47:31.029331 (+ 81us) server_negotiation.cc:299] Negotiation successful
0924 13:47:31.029344 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260924 13:47:31.029577 19062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:31.028115 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.194:41447 (local address 127.13.149.193:35853)
0924 13:47:31.028311 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0924 13:47:31.028319 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0924 13:47:31.028358 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:47:31.028472 (+ 114us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:47:31.028473 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:47:31.028486 (+ 13us) client_negotiation.cc:189] Negotiated authn=SASL
0924 13:47:31.029082 (+ 596us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0924 13:47:31.029082 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0924 13:47:31.029088 (+ 6us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0924 13:47:31.029098 (+ 10us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 13:47:31.029272 (+ 174us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 13:47:31.029272 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 13:47:31.029273 (+ 1us) client_negotiation.cc:769] Sending connection context
0924 13:47:31.029485 (+ 212us) client_negotiation.cc:240] Negotiation successful
0924 13:47:31.029499 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":182,"mutex_wait_us":23,"thread_start_us":114,"threads_started":1}
I20260924 13:47:31.029911 18860 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fb9e5c3b2c784ca5aa7d5865b7da4194" candidate_uuid: "f4854a67282b4b43aa5e59ea38c69144" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80d29b3ad60748d6be19519e23c320c7" is_pre_election: true
I20260924 13:47:31.029711 18992 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fb9e5c3b2c784ca5aa7d5865b7da4194" candidate_uuid: "f4854a67282b4b43aa5e59ea38c69144" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "71dd31eab16645cd8e0104029a46c6f5" is_pre_election: true
I20260924 13:47:31.030061 18860 raft_consensus.cc:2454] T fb9e5c3b2c784ca5aa7d5865b7da4194 P 80d29b3ad60748d6be19519e23c320c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f4854a67282b4b43aa5e59ea38c69144 in term 0.
I20260924 13:47:31.030077 18992 raft_consensus.cc:2454] T fb9e5c3b2c784ca5aa7d5865b7da4194 P 71dd31eab16645cd8e0104029a46c6f5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f4854a67282b4b43aa5e59ea38c69144 in term 0.
I20260924 13:47:31.030319 18663 leader_election.cc:302] T fb9e5c3b2c784ca5aa7d5865b7da4194 P f4854a67282b4b43aa5e59ea38c69144 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 80d29b3ad60748d6be19519e23c320c7, f4854a67282b4b43aa5e59ea38c69144; no voters:
I20260924 13:47:31.030463 19059 raft_consensus.cc:2781] T fb9e5c3b2c784ca5aa7d5865b7da4194 P f4854a67282b4b43aa5e59ea38c69144 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 13:47:31.030524 19059 raft_consensus.cc:482] T fb9e5c3b2c784ca5aa7d5865b7da4194 P f4854a67282b4b43aa5e59ea38c69144 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 13:47:31.030555 19059 raft_consensus.cc:3037] T fb9e5c3b2c784ca5aa7d5865b7da4194 P f4854a67282b4b43aa5e59ea38c69144 [term 0 FOLLOWER]: Advancing to term 1
I20260924 13:47:31.031277 19059 raft_consensus.cc:504] T fb9e5c3b2c784ca5aa7d5865b7da4194 P f4854a67282b4b43aa5e59ea38c69144 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4854a67282b4b43aa5e59ea38c69144" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 40315 } } peers { permanent_uuid: "71dd31eab16645cd8e0104029a46c6f5" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 37641 } } peers { permanent_uuid: "80d29b3ad60748d6be19519e23c320c7" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 41447 } }
I20260924 13:47:31.031425 19059 leader_election.cc:288] T fb9e5c3b2c784ca5aa7d5865b7da4194 P f4854a67282b4b43aa5e59ea38c69144 [CANDIDATE]: Term 1 election: Requested vote from peers 71dd31eab16645cd8e0104029a46c6f5 (127.13.149.195:37641), 80d29b3ad60748d6be19519e23c320c7 (127.13.149.194:41447)
I20260924 13:47:31.031737 18860 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fb9e5c3b2c784ca5aa7d5865b7da4194" candidate_uuid: "f4854a67282b4b43aa5e59ea38c69144" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80d29b3ad60748d6be19519e23c320c7"
I20260924 13:47:31.031831 18860 raft_consensus.cc:3037] T fb9e5c3b2c784ca5aa7d5865b7da4194 P 80d29b3ad60748d6be19519e23c320c7 [term 0 FOLLOWER]: Advancing to term 1
I20260924 13:47:31.031809 18992 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fb9e5c3b2c784ca5aa7d5865b7da4194" candidate_uuid: "f4854a67282b4b43aa5e59ea38c69144" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "71dd31eab16645cd8e0104029a46c6f5"
I20260924 13:47:31.031915 18992 raft_consensus.cc:3037] T fb9e5c3b2c784ca5aa7d5865b7da4194 P 71dd31eab16645cd8e0104029a46c6f5 [term 0 FOLLOWER]: Advancing to term 1
I20260924 13:47:31.032619 18860 raft_consensus.cc:2454] T fb9e5c3b2c784ca5aa7d5865b7da4194 P 80d29b3ad60748d6be19519e23c320c7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f4854a67282b4b43aa5e59ea38c69144 in term 1.
I20260924 13:47:31.032685 18992 raft_consensus.cc:2454] T fb9e5c3b2c784ca5aa7d5865b7da4194 P 71dd31eab16645cd8e0104029a46c6f5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f4854a67282b4b43aa5e59ea38c69144 in term 1.
I20260924 13:47:31.032821 18663 leader_election.cc:302] T fb9e5c3b2c784ca5aa7d5865b7da4194 P f4854a67282b4b43aa5e59ea38c69144 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 80d29b3ad60748d6be19519e23c320c7, f4854a67282b4b43aa5e59ea38c69144; no voters:
I20260924 13:47:31.032953 19059 raft_consensus.cc:2781] T fb9e5c3b2c784ca5aa7d5865b7da4194 P f4854a67282b4b43aa5e59ea38c69144 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 13:47:31.033139 19059 raft_consensus.cc:686] T fb9e5c3b2c784ca5aa7d5865b7da4194 P f4854a67282b4b43aa5e59ea38c69144 [term 1 LEADER]: Becoming Leader. State: Replica: f4854a67282b4b43aa5e59ea38c69144, State: Running, Role: LEADER
I20260924 13:47:31.033248 19059 consensus_queue.cc:237] T fb9e5c3b2c784ca5aa7d5865b7da4194 P f4854a67282b4b43aa5e59ea38c69144 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4854a67282b4b43aa5e59ea38c69144" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 40315 } } peers { permanent_uuid: "71dd31eab16645cd8e0104029a46c6f5" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 37641 } } peers { permanent_uuid: "80d29b3ad60748d6be19519e23c320c7" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 41447 } }
I20260924 13:47:31.034144 18583 catalog_manager.cc:5495] T fb9e5c3b2c784ca5aa7d5865b7da4194 P f4854a67282b4b43aa5e59ea38c69144 reported cstate change: term changed from 0 to 1, leader changed from <none> to f4854a67282b4b43aa5e59ea38c69144 (127.13.149.193). New cstate: current_term: 1 leader_uuid: "f4854a67282b4b43aa5e59ea38c69144" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4854a67282b4b43aa5e59ea38c69144" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 40315 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "71dd31eab16645cd8e0104029a46c6f5" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 37641 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "80d29b3ad60748d6be19519e23c320c7" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 41447 } health_report { overall_health: UNKNOWN } } }
W20260924 13:47:31.068142 18774 tablet.cc:2369] T fb9e5c3b2c784ca5aa7d5865b7da4194 P f4854a67282b4b43aa5e59ea38c69144: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260924 13:47:31.075454 19040 tablet.cc:2369] T fb9e5c3b2c784ca5aa7d5865b7da4194 P 71dd31eab16645cd8e0104029a46c6f5: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260924 13:47:31.076993 19049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:31.076089 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48388 (local address 127.13.149.193:40315)
0924 13:47:31.076140 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0924 13:47:31.076142 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:47:31.076155 (+ 13us) server_negotiation.cc:407] Connection header received
0924 13:47:31.076195 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:47:31.076196 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:47:31.076210 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:47:31.076265 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:47:31.076790 (+ 525us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:47:31.076791 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:47:31.076791 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 13:47:31.076795 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:47:31.076811 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 13:47:31.076815 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:47:31.076858 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0924 13:47:31.076892 (+ 34us) server_negotiation.cc:299] Negotiation successful
0924 13:47:31.076903 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260924 13:47:31.078655 18860 raft_consensus.cc:1261] T fb9e5c3b2c784ca5aa7d5865b7da4194 P 80d29b3ad60748d6be19519e23c320c7 [term 1 FOLLOWER]: Refusing update from remote peer f4854a67282b4b43aa5e59ea38c69144: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 13:47:31.078681 18992 raft_consensus.cc:1261] T fb9e5c3b2c784ca5aa7d5865b7da4194 P 71dd31eab16645cd8e0104029a46c6f5 [term 1 FOLLOWER]: Refusing update from remote peer f4854a67282b4b43aa5e59ea38c69144: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 13:47:31.078985 19059 consensus_queue.cc:1035] T fb9e5c3b2c784ca5aa7d5865b7da4194 P f4854a67282b4b43aa5e59ea38c69144 [LEADER]: Connected to new peer: Peer: permanent_uuid: "71dd31eab16645cd8e0104029a46c6f5" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 37641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 13:47:31.079123 19063 consensus_queue.cc:1035] T fb9e5c3b2c784ca5aa7d5865b7da4194 P f4854a67282b4b43aa5e59ea38c69144 [LEADER]: Connected to new peer: Peer: permanent_uuid: "80d29b3ad60748d6be19519e23c320c7" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 41447 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 13:47:31.082368 19068 mvcc.cc:205] Tried to move back new op lower bound from 7332895338815741952 to 7332895338632884224. Current Snapshot: MvccSnapshot[applied={T|T < 7332895338815741952}]
I20260924 13:47:31.082624 19067 mvcc.cc:205] Tried to move back new op lower bound from 7332895338815741952 to 7332895338632884224. Current Snapshot: MvccSnapshot[applied={T|T < 7332895338815741952}]
I20260924 13:47:31.084782 18584 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44020:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 13:47:31.084883 18584 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44020:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 13:47:31.085677 18584 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2067f395baab4c38bb51745a438d5c61: Sending DeleteTablet for 3 replicas of tablet fb9e5c3b2c784ca5aa7d5865b7da4194
I20260924 13:47:31.086069 18972 tablet_service.cc:1514] Processing DeleteTablet for tablet fb9e5c3b2c784ca5aa7d5865b7da4194 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 13:47:31 UTC) from {username='slave'} at 127.0.0.1:56142
I20260924 13:47:31.086207 13911 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJN9WRF/build/release/bin/kudu with pid 18645
I20260924 13:47:31.086297 18707 tablet_service.cc:1514] Processing DeleteTablet for tablet fb9e5c3b2c784ca5aa7d5865b7da4194 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 13:47:31 UTC) from {username='slave'} at 127.0.0.1:48376
I20260924 13:47:31.088101 18840 tablet_service.cc:1514] Processing DeleteTablet for tablet fb9e5c3b2c784ca5aa7d5865b7da4194 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 13:47:31 UTC) from {username='slave'} at 127.0.0.1:42084
I20260924 13:47:31.088361 19077 tablet_replica.cc:330] T fb9e5c3b2c784ca5aa7d5865b7da4194 P 80d29b3ad60748d6be19519e23c320c7: stopping tablet replica
I20260924 13:47:31.088492 19077 raft_consensus.cc:2229] T fb9e5c3b2c784ca5aa7d5865b7da4194 P 80d29b3ad60748d6be19519e23c320c7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 13:47:31.088608 19077 raft_consensus.cc:2258] T fb9e5c3b2c784ca5aa7d5865b7da4194 P 80d29b3ad60748d6be19519e23c320c7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 13:47:31.089035 19076 tablet_replica.cc:330] T fb9e5c3b2c784ca5aa7d5865b7da4194 P 71dd31eab16645cd8e0104029a46c6f5: stopping tablet replica
I20260924 13:47:31.089107 19077 ts_tablet_manager.cc:1905] T fb9e5c3b2c784ca5aa7d5865b7da4194 P 80d29b3ad60748d6be19519e23c320c7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 13:47:31.089157 19076 raft_consensus.cc:2229] T fb9e5c3b2c784ca5aa7d5865b7da4194 P 71dd31eab16645cd8e0104029a46c6f5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 13:47:31.089319 19076 raft_consensus.cc:2258] T fb9e5c3b2c784ca5aa7d5865b7da4194 P 71dd31eab16645cd8e0104029a46c6f5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 13:47:31.089772 19076 ts_tablet_manager.cc:1905] T fb9e5c3b2c784ca5aa7d5865b7da4194 P 71dd31eab16645cd8e0104029a46c6f5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 13:47:31.091180 19077 ts_tablet_manager.cc:1918] T fb9e5c3b2c784ca5aa7d5865b7da4194 P 80d29b3ad60748d6be19519e23c320c7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 13:47:31.091190 19076 ts_tablet_manager.cc:1918] T fb9e5c3b2c784ca5aa7d5865b7da4194 P 71dd31eab16645cd8e0104029a46c6f5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 13:47:31.091265 19076 log.cc:1196] T fb9e5c3b2c784ca5aa7d5865b7da4194 P 71dd31eab16645cd8e0104029a46c6f5: Deleting WAL directory at /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790257629041645-13911-0/minicluster-data/ts-2/wal/wals/fb9e5c3b2c784ca5aa7d5865b7da4194
I20260924 13:47:31.091271 19077 log.cc:1196] T fb9e5c3b2c784ca5aa7d5865b7da4194 P 80d29b3ad60748d6be19519e23c320c7: Deleting WAL directory at /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790257629041645-13911-0/minicluster-data/ts-1/wal/wals/fb9e5c3b2c784ca5aa7d5865b7da4194
I20260924 13:47:31.091511 19077 ts_tablet_manager.cc:1939] T fb9e5c3b2c784ca5aa7d5865b7da4194 P 80d29b3ad60748d6be19519e23c320c7: Deleting consensus metadata
I20260924 13:47:31.091531 19076 ts_tablet_manager.cc:1939] T fb9e5c3b2c784ca5aa7d5865b7da4194 P 71dd31eab16645cd8e0104029a46c6f5: Deleting consensus metadata
I20260924 13:47:31.091965 18571 catalog_manager.cc:4841] TS 80d29b3ad60748d6be19519e23c320c7 (127.13.149.194:41447): tablet fb9e5c3b2c784ca5aa7d5865b7da4194 (table test-table [id=667e10491d5b4de9b33c3ce7b1fdab29]) successfully deleted
I20260924 13:47:31.092096 18569 catalog_manager.cc:4841] TS 71dd31eab16645cd8e0104029a46c6f5 (127.13.149.195:37641): tablet fb9e5c3b2c784ca5aa7d5865b7da4194 (table test-table [id=667e10491d5b4de9b33c3ce7b1fdab29]) successfully deleted
W20260924 13:47:31.092346 18571 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.13.149.193:40315 (error 108)
I20260924 13:47:31.092475 13911 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJN9WRF/build/release/bin/kudu with pid 18776
I20260924 13:47:31.093235 19048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:31.093097 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.193:40315 (local address 127.0.0.1:48390)
0924 13:47:31.093148 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0924 13:47:31.093201 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.149.193:40315: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260924 13:47:31.093346 18571 catalog_manager.cc:4584] TS f4854a67282b4b43aa5e59ea38c69144 (127.13.149.193:40315): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet fb9e5c3b2c784ca5aa7d5865b7da4194: Network error: Client connection negotiation failed: client connection to 127.13.149.193:40315: connect: Connection refused (error 111)
I20260924 13:47:31.096968 13911 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJN9WRF/build/release/bin/kudu with pid 18910
I20260924 13:47:31.101352 13911 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJN9WRF/build/release/bin/kudu with pid 18553
2026-09-24T13:47:31Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (839 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260924 13:47:31.120543 13911 mini_oidc.cc:152] Starting JWKS server
I20260924 13:47:31.121738 13911 webserver.cc:460] Webserver started at https://127.0.0.1:33043/ using document root <none> and password file <none>
I20260924 13:47:31.121827 13911 mini_oidc.cc:189] Starting OIDC Discovery server
I20260924 13:47:31.121991 13911 webserver.cc:460] Webserver started at http://127.0.0.1:36031/ using document root <none> and password file <none>
2026-09-24T13:47:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T13:47:31Z Disabled control of system clock
I20260924 13:47:31.129334 13911 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJN9WRF/build/release/bin/kudu
/tmp/dist-test-taskJN9WRF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790257629041645-13911-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790257629041645-13911-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790257629041645-13911-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790257629041645-13911-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.149.254:42347
--webserver_interface=127.13.149.254
--webserver_port=0
--builtin_ntp_servers=127.13.149.212:46079
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.149.254:42347
--enable_jwt_token_auth=true
--jwks_url=https://localhost:33043/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790257629041645-13911-0/testchainca.cert with env {}
W20260924 13:47:31.210984 19084 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 13:47:31.211220 19084 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 13:47:31.211243 19084 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 13:47:31.212435 19084 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 13:47:31.212476 19084 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 13:47:31.212486 19084 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 13:47:31.212498 19084 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 13:47:31.212508 19084 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260924 13:47:31.212520 19084 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260924 13:47:31.212534 19084 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:33043/jwks.json
I20260924 13:47:31.214229 19084 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.149.212:46079
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790257629041645-13911-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790257629041645-13911-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.149.254:42347
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.149.254:42347
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:33043/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790257629041645-13911-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.149.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790257629041645-13911-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19084
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790257629041645-13911-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 12:29:37 UTC on ccycloud.quasar-iqzbxn.root.comops.site
build id 4670106
I20260924 13:47:31.214461 19084 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 13:47:31.214646 19084 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260924 13:47:31.219763 19084 server_base.cc:1034] running on GCE node
W20260924 13:47:31.219677 19090 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 13:47:31.219674 19089 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 13:47:31.219726 19092 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 13:47:31.220362 19084 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 13:47:31.220693 19084 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 13:47:31.221868 19084 hybrid_clock.cc:648] HybridClock initialized: now 1790257651221851 us; error 29 us; skew 500 ppm
I20260924 13:47:31.223321 19084 webserver.cc:460] Webserver started at http://127.13.149.254:39737/ using document root <none> and password file <none>
I20260924 13:47:31.223577 19084 fs_manager.cc:359] Metadata directory not provided
I20260924 13:47:31.223681 19084 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 13:47:31.223810 19084 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 13:47:31.224907 19084 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790257629041645-13911-0/minicluster-data/master-0/data/instance:
uuid: "e3702b2a4d1a4186ac69127811fab39f"
format_stamp: "Formatted at 2026-09-24 13:47:31 on dist-test-slave-sjl2"
server_key: "2c15669d71e40115723e027bbe6cd7ad"
server_key_iv: "a734ee1a5a3d1306b2ad542960d38c2c"
server_key_version: "encryptionkey@0"
I20260924 13:47:31.225313 19084 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790257629041645-13911-0/minicluster-data/master-0/wal/instance:
uuid: "e3702b2a4d1a4186ac69127811fab39f"
format_stamp: "Formatted at 2026-09-24 13:47:31 on dist-test-slave-sjl2"
server_key: "2c15669d71e40115723e027bbe6cd7ad"
server_key_iv: "a734ee1a5a3d1306b2ad542960d38c2c"
server_key_version: "encryptionkey@0"
I20260924 13:47:31.226601 19084 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260924 13:47:31.227438 19098 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 13:47:31.227602 19084 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260924 13:47:31.227739 19084 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790257629041645-13911-0/minicluster-data/master-0/data,/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790257629041645-13911-0/minicluster-data/master-0/wal
uuid: "e3702b2a4d1a4186ac69127811fab39f"
format_stamp: "Formatted at 2026-09-24 13:47:31 on dist-test-slave-sjl2"
server_key: "2c15669d71e40115723e027bbe6cd7ad"
server_key_iv: "a734ee1a5a3d1306b2ad542960d38c2c"
server_key_version: "encryptionkey@0"
I20260924 13:47:31.227813 19084 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790257629041645-13911-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790257629041645-13911-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790257629041645-13911-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 13:47:31.241712 19084 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 13:47:31.242036 19084 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 13:47:31.242169 19084 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 13:47:31.246102 19084 rpc_server.cc:307] RPC server started. Bound to: 127.13.149.254:42347
I20260924 13:47:31.246141 19150 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.149.254:42347 every 8 connection(s)
I20260924 13:47:31.246483 19084 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790257629041645-13911-0/minicluster-data/master-0/data/info.pb
I20260924 13:47:31.247452 19151 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 13:47:31.249948 19151 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e3702b2a4d1a4186ac69127811fab39f: Bootstrap starting.
I20260924 13:47:31.250548 19151 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e3702b2a4d1a4186ac69127811fab39f: Neither blocks nor log segments found. Creating new log.
I20260924 13:47:31.250798 19151 log.cc:824] T 00000000000000000000000000000000 P e3702b2a4d1a4186ac69127811fab39f: Log is configured to *not* fsync() on all Append() calls
I20260924 13:47:31.251549 19151 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e3702b2a4d1a4186ac69127811fab39f: No bootstrap required, opened a new log
I20260924 13:47:31.253409 19151 raft_consensus.cc:348] T 00000000000000000000000000000000 P e3702b2a4d1a4186ac69127811fab39f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e3702b2a4d1a4186ac69127811fab39f" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 42347 } }
I20260924 13:47:31.253582 19151 raft_consensus.cc:374] T 00000000000000000000000000000000 P e3702b2a4d1a4186ac69127811fab39f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 13:47:31.253618 19151 raft_consensus.cc:729] T 00000000000000000000000000000000 P e3702b2a4d1a4186ac69127811fab39f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3702b2a4d1a4186ac69127811fab39f, State: Initialized, Role: FOLLOWER
I20260924 13:47:31.253813 19151 consensus_queue.cc:260] T 00000000000000000000000000000000 P e3702b2a4d1a4186ac69127811fab39f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e3702b2a4d1a4186ac69127811fab39f" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 42347 } }
I20260924 13:47:31.253912 19151 raft_consensus.cc:388] T 00000000000000000000000000000000 P e3702b2a4d1a4186ac69127811fab39f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 13:47:31.253952 19151 raft_consensus.cc:482] T 00000000000000000000000000000000 P e3702b2a4d1a4186ac69127811fab39f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 13:47:31.253931 13911 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJN9WRF/build/release/bin/kudu as pid 19084
I20260924 13:47:31.254004 19151 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e3702b2a4d1a4186ac69127811fab39f [term 0 FOLLOWER]: Advancing to term 1
I20260924 13:47:31.254036 13911 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790257629041645-13911-0/minicluster-data/master-0/wal/instance
I20260924 13:47:31.254213 13911 external_mini_cluster.cc:1428] Setting key 063f4cb75bce2b3f581428519446fd87
I20260924 13:47:31.254737 19151 raft_consensus.cc:504] T 00000000000000000000000000000000 P e3702b2a4d1a4186ac69127811fab39f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e3702b2a4d1a4186ac69127811fab39f" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 42347 } }
I20260924 13:47:31.254870 19151 leader_election.cc:302] T 00000000000000000000000000000000 P e3702b2a4d1a4186ac69127811fab39f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e3702b2a4d1a4186ac69127811fab39f; no voters:
I20260924 13:47:31.255039 19151 leader_election.cc:288] T 00000000000000000000000000000000 P e3702b2a4d1a4186ac69127811fab39f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 13:47:31.255075 19156 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e3702b2a4d1a4186ac69127811fab39f [term 1 FOLLOWER]: Leader election won for term 1
I20260924 13:47:31.255250 19156 raft_consensus.cc:686] T 00000000000000000000000000000000 P e3702b2a4d1a4186ac69127811fab39f [term 1 LEADER]: Becoming Leader. State: Replica: e3702b2a4d1a4186ac69127811fab39f, State: Running, Role: LEADER
I20260924 13:47:31.255307 19151 sys_catalog.cc:564] T 00000000000000000000000000000000 P e3702b2a4d1a4186ac69127811fab39f [sys.catalog]: configured and running, proceeding with master startup.
I20260924 13:47:31.255358 19156 consensus_queue.cc:237] T 00000000000000000000000000000000 P e3702b2a4d1a4186ac69127811fab39f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e3702b2a4d1a4186ac69127811fab39f" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 42347 } }
I20260924 13:47:31.255723 19156 sys_catalog.cc:455] T 00000000000000000000000000000000 P e3702b2a4d1a4186ac69127811fab39f [sys.catalog]: SysCatalogTable state changed. Reason: New leader e3702b2a4d1a4186ac69127811fab39f. Latest consensus state: current_term: 1 leader_uuid: "e3702b2a4d1a4186ac69127811fab39f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e3702b2a4d1a4186ac69127811fab39f" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 42347 } } }
I20260924 13:47:31.255797 19156 sys_catalog.cc:458] T 00000000000000000000000000000000 P e3702b2a4d1a4186ac69127811fab39f [sys.catalog]: This master's current role is: LEADER
I20260924 13:47:31.256014 19160 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 13:47:31.256558 19160 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 13:47:31.257243 19157 sys_catalog.cc:455] T 00000000000000000000000000000000 P e3702b2a4d1a4186ac69127811fab39f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e3702b2a4d1a4186ac69127811fab39f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e3702b2a4d1a4186ac69127811fab39f" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 42347 } } }
I20260924 13:47:31.257405 19157 sys_catalog.cc:458] T 00000000000000000000000000000000 P e3702b2a4d1a4186ac69127811fab39f [sys.catalog]: This master's current role is: LEADER
I20260924 13:47:31.258740 19160 catalog_manager.cc:1347] Generated new cluster ID: 403243db1d1d46d687c964b4d44f7d04
I20260924 13:47:31.258800 19160 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 13:47:31.260298 19155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:31.254720 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58550 (local address 127.13.149.254:42347)
0924 13:47:31.255001 (+ 281us) server_negotiation.cc:206] Beginning negotiation
0924 13:47:31.255010 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0924 13:47:31.255044 (+ 34us) server_negotiation.cc:407] Connection header received
0924 13:47:31.255174 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:47:31.255182 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:47:31.255254 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:47:31.255445 (+ 191us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:47:31.256061 (+ 616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:31.257504 (+ 1443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:31.258324 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:31.258929 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:31.259044 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:47:31.259675 (+ 631us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:47:31.259680 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:47:31.259680 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 13:47:31.259855 (+ 175us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:47:31.259889 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 13:47:31.259900 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:47:31.259957 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0924 13:47:31.260029 (+ 72us) server_negotiation.cc:299] Negotiation successful
0924 13:47:31.260109 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":197,"thread_start_us":109,"threads_started":1}
I20260924 13:47:31.265503 13911 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260924 13:47:31.271113 19160 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 13:47:31.271667 19160 catalog_manager.cc:1504] Loading token signing keys...
I20260924 13:47:31.277843 19160 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e3702b2a4d1a4186ac69127811fab39f: Generated new TSK 0
I20260924 13:47:31.278072 19160 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260924 13:47:31.305986 19182 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260924 13:47:31.306075 19182 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260924 13:47:31.306432 19155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:31.290360 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58554 (local address 127.13.149.254:42347)
0924 13:47:31.290412 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0924 13:47:31.290414 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:47:31.290493 (+ 79us) server_negotiation.cc:407] Connection header received
0924 13:47:31.290590 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:47:31.290591 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:47:31.290603 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:47:31.290688 (+ 85us) server_negotiation.cc:226] Negotiated authn=JWT
0924 13:47:31.290815 (+ 127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:31.291599 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:31.292369 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:31.292656 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:31.292728 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:47:31.292951 (+ 223us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0924 13:47:31.306131 (+ 13180us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33043/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0924 13:47:31.306336 (+ 205us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:58554: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33043/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":28}
W20260924 13:47:31.306493 19155 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:58554: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33043/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260924 13:47:31.306740 19181 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 13:47:31.290260 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:42347 (local address 127.0.0.1:58554)
0924 13:47:31.290438 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0924 13:47:31.290448 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 13:47:31.290509 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:47:31.290688 (+ 179us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:47:31.290688 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:47:31.290690 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0924 13:47:31.290752 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:47:31.290754 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:47:31.291687 (+ 933us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:47:31.291688 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:47:31.292291 (+ 603us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:47:31.292293 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:47:31.292736 (+ 443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:47:31.292737 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:47:31.292781 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:47:31.292803 (+ 22us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0924 13:47:31.306307 (+ 13504us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33043/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0924 13:47:31.306410 (+ 103us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.13.149.254:42347: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33043/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":83,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.13.149.254:42347: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33043/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260924 13:47:31.307251 13911 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJN9WRF/build/release/bin/kudu with pid 19084
2026-09-24T13:47:31Z chronyd exiting
I20260924 13:47:31.342861 13911 test_util.cc:175] -----------------------------------------------
I20260924 13:47:31.342949 13911 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790257629041645-13911-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (223 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260924 13:47:31.343760 13911 mini_oidc.cc:152] Starting JWKS server
I20260924 13:47:31.344444 13911 webserver.cc:460] Webserver started at https://127.0.0.1:39989/ using document root <none> and password file <none>
I20260924 13:47:31.344489 13911 mini_oidc.cc:189] Starting OIDC Discovery server
I20260924 13:47:31.344627 13911 webserver.cc:460] Webserver started at http://127.0.0.1:45731/ using document root <none> and password file <none>
2026-09-24T13:47:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T13:47:31Z Disabled control of system clock
I20260924 13:47:31.352035 13911 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJN9WRF/build/release/bin/kudu
/tmp/dist-test-taskJN9WRF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790257629041645-13911-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790257629041645-13911-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790257629041645-13911-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790257629041645-13911-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.149.254:33335
--webserver_interface=127.13.149.254
--webserver_port=0
--builtin_ntp_servers=127.13.149.212:38597
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.149.254:33335
--enable_jwt_token_auth=true
--jwks_url=https://localhost:39989/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790257629041645-13911-0/testchainca.cert with env {}
W20260924 13:47:31.434067 19188 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 13:47:31.434293 19188 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 13:47:31.434317 19188 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 13:47:31.435599 19188 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 13:47:31.435662 19188 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 13:47:31.435674 19188 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 13:47:31.435686 19188 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 13:47:31.435698 19188 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260924 13:47:31.435710 19188 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260924 13:47:31.435734 19188 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:39989/jwks.json
I20260924 13:47:31.437433 19188 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.149.212:38597
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790257629041645-13911-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790257629041645-13911-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.149.254:33335
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.149.254:33335
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:39989/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790257629041645-13911-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.149.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790257629041645-13911-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19188
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790257629041645-13911-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 12:29:37 UTC on ccycloud.quasar-iqzbxn.root.comops.site
build id 4670106
I20260924 13:47:31.437702 19188 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 13:47:31.437919 19188 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 13:47:31.442919 19193 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 13:47:31.442920 19196 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 13:47:31.443015 19188 server_base.cc:1034] running on GCE node
W20260924 13:47:31.442926 19194 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 13:47:31.443430 19188 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 13:47:31.443745 19188 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 13:47:31.444954 19188 hybrid_clock.cc:648] HybridClock initialized: now 1790257651444929 us; error 31 us; skew 500 ppm
I20260924 13:47:31.446285 19188 webserver.cc:460] Webserver started at http://127.13.149.254:38083/ using document root <none> and password file <none>
I20260924 13:47:31.446524 19188 fs_manager.cc:359] Metadata directory not provided
I20260924 13:47:31.446573 19188 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 13:47:31.446703 19188 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 13:47:31.447830 19188 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790257629041645-13911-0/minicluster-data/master-0/data/instance:
uuid: "e660727bc1944963ad4a35464a97fa00"
format_stamp: "Formatted at 2026-09-24 13:47:31 on dist-test-slave-sjl2"
server_key: "3a3050db07d63072770cba59134d4234"
server_key_iv: "3cf6ccbfe776d2302c6ed607cae339f1"
server_key_version: "encryptionkey@0"
I20260924 13:47:31.448205 19188 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790257629041645-13911-0/minicluster-data/master-0/wal/instance:
uuid: "e660727bc1944963ad4a35464a97fa00"
format_stamp: "Formatted at 2026-09-24 13:47:31 on dist-test-slave-sjl2"
server_key: "3a3050db07d63072770cba59134d4234"
server_key_iv: "3cf6ccbfe776d2302c6ed607cae339f1"
server_key_version: "encryptionkey@0"
I20260924 13:47:31.449844 19188 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260924 13:47:31.450773 19202 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 13:47:31.450968 19188 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260924 13:47:31.451069 19188 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790257629041645-13911-0/minicluster-data/master-0/data,/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790257629041645-13911-0/minicluster-data/master-0/wal
uuid: "e660727bc1944963ad4a35464a97fa00"
format_stamp: "Formatted at 2026-09-24 13:47:31 on dist-test-slave-sjl2"
server_key: "3a3050db07d63072770cba59134d4234"
server_key_iv: "3cf6ccbfe776d2302c6ed607cae339f1"
server_key_version: "encryptionkey@0"
I20260924 13:47:31.451157 19188 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790257629041645-13911-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790257629041645-13911-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790257629041645-13911-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 13:47:31.476225 19188 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 13:47:31.476567 19188 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 13:47:31.476711 19188 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 13:47:31.480772 19188 rpc_server.cc:307] RPC server started. Bound to: 127.13.149.254:33335
I20260924 13:47:31.480827 19254 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.149.254:33335 every 8 connection(s)
I20260924 13:47:31.481176 19188 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790257629041645-13911-0/minicluster-data/master-0/data/info.pb
I20260924 13:47:31.481927 19255 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 13:47:31.485023 19255 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e660727bc1944963ad4a35464a97fa00: Bootstrap starting.
I20260924 13:47:31.485682 19255 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e660727bc1944963ad4a35464a97fa00: Neither blocks nor log segments found. Creating new log.
I20260924 13:47:31.485965 19255 log.cc:824] T 00000000000000000000000000000000 P e660727bc1944963ad4a35464a97fa00: Log is configured to *not* fsync() on all Append() calls
I20260924 13:47:31.486791 13911 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJN9WRF/build/release/bin/kudu as pid 19188
I20260924 13:47:31.486837 19255 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e660727bc1944963ad4a35464a97fa00: No bootstrap required, opened a new log
I20260924 13:47:31.486894 13911 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790257629041645-13911-0/minicluster-data/master-0/wal/instance
I20260924 13:47:31.487287 13911 external_mini_cluster.cc:1428] Setting key 101a7af12dfc1a585d2690733967681e
I20260924 13:47:31.489049 19255 raft_consensus.cc:348] T 00000000000000000000000000000000 P e660727bc1944963ad4a35464a97fa00 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e660727bc1944963ad4a35464a97fa00" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 33335 } }
I20260924 13:47:31.489229 19255 raft_consensus.cc:374] T 00000000000000000000000000000000 P e660727bc1944963ad4a35464a97fa00 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 13:47:31.489265 19255 raft_consensus.cc:729] T 00000000000000000000000000000000 P e660727bc1944963ad4a35464a97fa00 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e660727bc1944963ad4a35464a97fa00, State: Initialized, Role: FOLLOWER
I20260924 13:47:31.489418 19255 consensus_queue.cc:260] T 00000000000000000000000000000000 P e660727bc1944963ad4a35464a97fa00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e660727bc1944963ad4a35464a97fa00" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 33335 } }
I20260924 13:47:31.489492 19255 raft_consensus.cc:388] T 00000000000000000000000000000000 P e660727bc1944963ad4a35464a97fa00 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 13:47:31.489528 19255 raft_consensus.cc:482] T 00000000000000000000000000000000 P e660727bc1944963ad4a35464a97fa00 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 13:47:31.489566 19255 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e660727bc1944963ad4a35464a97fa00 [term 0 FOLLOWER]: Advancing to term 1
I20260924 13:47:31.493662 19259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:31.487870 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54974 (local address 127.13.149.254:33335)
0924 13:47:31.488103 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0924 13:47:31.488111 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0924 13:47:31.488140 (+ 29us) server_negotiation.cc:407] Connection header received
0924 13:47:31.489028 (+ 888us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:47:31.489038 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:47:31.489130 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:47:31.489281 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:47:31.489528 (+ 247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:31.490953 (+ 1425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:31.491746 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:31.492179 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:31.492301 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:47:31.493003 (+ 702us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:47:31.493007 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:47:31.493007 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 13:47:31.493198 (+ 191us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:47:31.493237 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 13:47:31.493248 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:47:31.493306 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0924 13:47:31.493395 (+ 89us) server_negotiation.cc:299] Negotiation successful
0924 13:47:31.493490 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":147,"thread_start_us":93,"threads_started":1}
I20260924 13:47:31.494832 13911 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260924 13:47:31.518764 19255 raft_consensus.cc:504] T 00000000000000000000000000000000 P e660727bc1944963ad4a35464a97fa00 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e660727bc1944963ad4a35464a97fa00" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 33335 } }
I20260924 13:47:31.518958 19255 leader_election.cc:302] T 00000000000000000000000000000000 P e660727bc1944963ad4a35464a97fa00 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e660727bc1944963ad4a35464a97fa00; no voters:
I20260924 13:47:31.519204 19255 leader_election.cc:288] T 00000000000000000000000000000000 P e660727bc1944963ad4a35464a97fa00 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 13:47:31.519413 19260 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e660727bc1944963ad4a35464a97fa00 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 13:47:31.519562 19255 sys_catalog.cc:564] T 00000000000000000000000000000000 P e660727bc1944963ad4a35464a97fa00 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 13:47:31.519721 19260 raft_consensus.cc:686] T 00000000000000000000000000000000 P e660727bc1944963ad4a35464a97fa00 [term 1 LEADER]: Becoming Leader. State: Replica: e660727bc1944963ad4a35464a97fa00, State: Running, Role: LEADER
I20260924 13:47:31.520066 19260 consensus_queue.cc:237] T 00000000000000000000000000000000 P e660727bc1944963ad4a35464a97fa00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e660727bc1944963ad4a35464a97fa00" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 33335 } }
I20260924 13:47:31.520836 19261 sys_catalog.cc:455] T 00000000000000000000000000000000 P e660727bc1944963ad4a35464a97fa00 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e660727bc1944963ad4a35464a97fa00" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e660727bc1944963ad4a35464a97fa00" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 33335 } } }
I20260924 13:47:31.520972 19261 sys_catalog.cc:458] T 00000000000000000000000000000000 P e660727bc1944963ad4a35464a97fa00 [sys.catalog]: This master's current role is: LEADER
I20260924 13:47:31.521206 19262 sys_catalog.cc:455] T 00000000000000000000000000000000 P e660727bc1944963ad4a35464a97fa00 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e660727bc1944963ad4a35464a97fa00. Latest consensus state: current_term: 1 leader_uuid: "e660727bc1944963ad4a35464a97fa00" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e660727bc1944963ad4a35464a97fa00" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 33335 } } }
I20260924 13:47:31.521277 19262 sys_catalog.cc:458] T 00000000000000000000000000000000 P e660727bc1944963ad4a35464a97fa00 [sys.catalog]: This master's current role is: LEADER
W20260924 13:47:31.521956 19274 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e660727bc1944963ad4a35464a97fa00: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260924 13:47:31.522012 19274 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260924 13:47:31.522061 19275 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 13:47:31.522229 19275 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 13:47:31.523910 19275 catalog_manager.cc:1347] Generated new cluster ID: ad89e999da984f7481131cadd16fc458
I20260924 13:47:31.523964 19275 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 13:47:31.529816 19275 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 13:47:31.530460 19275 catalog_manager.cc:1504] Loading token signing keys...
I20260924 13:47:31.534137 19275 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e660727bc1944963ad4a35464a97fa00: Generated new TSK 0
I20260924 13:47:31.534366 19275 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260924 13:47:31.557443 19285 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260924 13:47:31.557528 19285 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260924 13:47:31.557942 19259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:31.542512 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54982 (local address 127.13.149.254:33335)
0924 13:47:31.542575 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0924 13:47:31.542577 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:47:31.542680 (+ 103us) server_negotiation.cc:407] Connection header received
0924 13:47:31.542762 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:47:31.542763 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:47:31.542778 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:47:31.542872 (+ 94us) server_negotiation.cc:226] Negotiated authn=JWT
0924 13:47:31.543072 (+ 200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:31.543940 (+ 868us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:31.544951 (+ 1011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:31.545272 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:31.545349 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:47:31.545562 (+ 213us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0924 13:47:31.557694 (+ 12132us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39989/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0924 13:47:31.557865 (+ 171us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:54982: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39989/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":30}
W20260924 13:47:31.558007 19259 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:54982: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39989/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260924 13:47:31.558099 19284 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 13:47:31.542426 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:33335 (local address 127.0.0.1:54982)
0924 13:47:31.542594 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0924 13:47:31.542605 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0924 13:47:31.542678 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:47:31.542911 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:47:31.542912 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:47:31.542915 (+ 3us) client_negotiation.cc:189] Negotiated authn=JWT
0924 13:47:31.542995 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:47:31.542997 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:47:31.544098 (+ 1101us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:47:31.544100 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:47:31.544813 (+ 713us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:47:31.544817 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:47:31.545384 (+ 567us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:47:31.545385 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:47:31.545440 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:47:31.545446 (+ 6us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0924 13:47:31.557845 (+ 12399us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39989/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0924 13:47:31.557973 (+ 128us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.13.149.254:33335: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39989/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":96,"threads_started":1}
I20260924 13:47:31.558398 13911 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJN9WRF/build/release/bin/kudu with pid 19188
2026-09-24T13:47:31Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (251 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260924 13:47:31.595731 13911 mini_oidc.cc:152] Starting JWKS server
I20260924 13:47:31.596455 13911 webserver.cc:460] Webserver started at https://127.0.0.1:40595/ using document root <none> and password file <none>
I20260924 13:47:31.596509 13911 mini_oidc.cc:189] Starting OIDC Discovery server
I20260924 13:47:31.596648 13911 webserver.cc:460] Webserver started at http://127.0.0.1:42079/ using document root <none> and password file <none>
2026-09-24T13:47:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T13:47:31Z Disabled control of system clock
I20260924 13:47:31.606228 13911 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJN9WRF/build/release/bin/kudu
/tmp/dist-test-taskJN9WRF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790257629041645-13911-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790257629041645-13911-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790257629041645-13911-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790257629041645-13911-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.149.254:33569
--webserver_interface=127.13.149.254
--webserver_port=0
--builtin_ntp_servers=127.13.149.212:42875
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.149.254:33569
--enable_jwt_token_auth=true
--jwks_url=https://localhost:40595/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260924 13:47:31.692409 19291 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 13:47:31.692656 19291 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 13:47:31.692689 19291 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 13:47:31.694068 19291 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 13:47:31.694132 19291 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 13:47:31.694144 19291 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 13:47:31.694156 19291 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 13:47:31.694166 19291 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260924 13:47:31.694180 19291 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260924 13:47:31.694192 19291 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:40595/jwks.json
I20260924 13:47:31.695771 19291 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.149.212:42875
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790257629041645-13911-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790257629041645-13911-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.149.254:33569
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.149.254:33569
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:40595/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790257629041645-13911-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.149.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19291
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790257629041645-13911-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 12:29:37 UTC on ccycloud.quasar-iqzbxn.root.comops.site
build id 4670106
I20260924 13:47:31.696048 19291 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 13:47:31.696363 19291 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 13:47:31.701421 19297 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 13:47:31.701421 19299 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 13:47:31.701421 19296 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 13:47:31.701997 19291 server_base.cc:1034] running on GCE node
I20260924 13:47:31.702234 19291 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 13:47:31.702554 19291 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 13:47:31.703784 19291 hybrid_clock.cc:648] HybridClock initialized: now 1790257651703756 us; error 36 us; skew 500 ppm
I20260924 13:47:31.705262 19291 webserver.cc:460] Webserver started at http://127.13.149.254:35689/ using document root <none> and password file <none>
I20260924 13:47:31.705497 19291 fs_manager.cc:359] Metadata directory not provided
I20260924 13:47:31.705549 19291 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 13:47:31.705722 19291 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 13:47:31.706866 19291 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790257629041645-13911-0/minicluster-data/master-0/data/instance:
uuid: "26099734212a41c385c13e1109372e8b"
format_stamp: "Formatted at 2026-09-24 13:47:31 on dist-test-slave-sjl2"
server_key: "b0013988b159fff2c61198c8c884ec01"
server_key_iv: "44e40ea71e96f58c027114bb7de4b707"
server_key_version: "encryptionkey@0"
I20260924 13:47:31.707223 19291 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790257629041645-13911-0/minicluster-data/master-0/wal/instance:
uuid: "26099734212a41c385c13e1109372e8b"
format_stamp: "Formatted at 2026-09-24 13:47:31 on dist-test-slave-sjl2"
server_key: "b0013988b159fff2c61198c8c884ec01"
server_key_iv: "44e40ea71e96f58c027114bb7de4b707"
server_key_version: "encryptionkey@0"
I20260924 13:47:31.708658 19291 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260924 13:47:31.709532 19305 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 13:47:31.709734 19291 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260924 13:47:31.709824 19291 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790257629041645-13911-0/minicluster-data/master-0/data,/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790257629041645-13911-0/minicluster-data/master-0/wal
uuid: "26099734212a41c385c13e1109372e8b"
format_stamp: "Formatted at 2026-09-24 13:47:31 on dist-test-slave-sjl2"
server_key: "b0013988b159fff2c61198c8c884ec01"
server_key_iv: "44e40ea71e96f58c027114bb7de4b707"
server_key_version: "encryptionkey@0"
I20260924 13:47:31.709896 19291 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790257629041645-13911-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790257629041645-13911-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790257629041645-13911-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 13:47:31.749459 19291 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 13:47:31.749850 19291 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 13:47:31.750010 19291 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 13:47:31.754011 19291 rpc_server.cc:307] RPC server started. Bound to: 127.13.149.254:33569
I20260924 13:47:31.754048 19357 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.149.254:33569 every 8 connection(s)
I20260924 13:47:31.754403 19291 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790257629041645-13911-0/minicluster-data/master-0/data/info.pb
I20260924 13:47:31.755169 19358 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 13:47:31.758249 19358 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 26099734212a41c385c13e1109372e8b: Bootstrap starting.
I20260924 13:47:31.758877 19358 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 26099734212a41c385c13e1109372e8b: Neither blocks nor log segments found. Creating new log.
I20260924 13:47:31.759138 19358 log.cc:824] T 00000000000000000000000000000000 P 26099734212a41c385c13e1109372e8b: Log is configured to *not* fsync() on all Append() calls
I20260924 13:47:31.759929 19358 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 26099734212a41c385c13e1109372e8b: No bootstrap required, opened a new log
I20260924 13:47:31.761832 13911 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJN9WRF/build/release/bin/kudu as pid 19291
I20260924 13:47:31.761931 13911 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790257629041645-13911-0/minicluster-data/master-0/wal/instance
I20260924 13:47:31.761829 19358 raft_consensus.cc:348] T 00000000000000000000000000000000 P 26099734212a41c385c13e1109372e8b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26099734212a41c385c13e1109372e8b" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 33569 } }
I20260924 13:47:31.762046 19358 raft_consensus.cc:374] T 00000000000000000000000000000000 P 26099734212a41c385c13e1109372e8b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 13:47:31.762082 19358 raft_consensus.cc:729] T 00000000000000000000000000000000 P 26099734212a41c385c13e1109372e8b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 26099734212a41c385c13e1109372e8b, State: Initialized, Role: FOLLOWER
I20260924 13:47:31.762143 13911 external_mini_cluster.cc:1428] Setting key 9a2b13a29b73d5d8ec3bb2e2e2aec62b
I20260924 13:47:31.762249 19358 consensus_queue.cc:260] T 00000000000000000000000000000000 P 26099734212a41c385c13e1109372e8b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26099734212a41c385c13e1109372e8b" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 33569 } }
I20260924 13:47:31.762360 19358 raft_consensus.cc:388] T 00000000000000000000000000000000 P 26099734212a41c385c13e1109372e8b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 13:47:31.762409 19358 raft_consensus.cc:482] T 00000000000000000000000000000000 P 26099734212a41c385c13e1109372e8b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 13:47:31.762486 19358 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 26099734212a41c385c13e1109372e8b [term 0 FOLLOWER]: Advancing to term 1
I20260924 13:47:31.768604 19363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:31.762786 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55398 (local address 127.13.149.254:33569)
0924 13:47:31.763035 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0924 13:47:31.763051 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0924 13:47:31.763108 (+ 57us) server_negotiation.cc:407] Connection header received
0924 13:47:31.763254 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:47:31.763261 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:47:31.763344 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:47:31.763465 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:47:31.763693 (+ 228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:31.764638 (+ 945us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:31.765647 (+ 1009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:31.766149 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:31.766260 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:47:31.767145 (+ 885us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:47:31.767150 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:47:31.767150 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 13:47:31.767367 (+ 217us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:47:31.767407 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 13:47:31.767421 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:47:31.767489 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0924 13:47:31.767881 (+ 392us) server_negotiation.cc:299] Negotiation successful
0924 13:47:31.768073 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":153,"thread_start_us":98,"threads_started":1}
I20260924 13:47:31.770328 13911 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260924 13:47:31.826655 19358 raft_consensus.cc:504] T 00000000000000000000000000000000 P 26099734212a41c385c13e1109372e8b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26099734212a41c385c13e1109372e8b" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 33569 } }
I20260924 13:47:31.826843 19358 leader_election.cc:302] T 00000000000000000000000000000000 P 26099734212a41c385c13e1109372e8b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 26099734212a41c385c13e1109372e8b; no voters:
I20260924 13:47:31.827060 19358 leader_election.cc:288] T 00000000000000000000000000000000 P 26099734212a41c385c13e1109372e8b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 13:47:31.827148 19364 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 26099734212a41c385c13e1109372e8b [term 1 FOLLOWER]: Leader election won for term 1
I20260924 13:47:31.827368 19364 raft_consensus.cc:686] T 00000000000000000000000000000000 P 26099734212a41c385c13e1109372e8b [term 1 LEADER]: Becoming Leader. State: Replica: 26099734212a41c385c13e1109372e8b, State: Running, Role: LEADER
I20260924 13:47:31.827399 19358 sys_catalog.cc:564] T 00000000000000000000000000000000 P 26099734212a41c385c13e1109372e8b [sys.catalog]: configured and running, proceeding with master startup.
I20260924 13:47:31.827481 19364 consensus_queue.cc:237] T 00000000000000000000000000000000 P 26099734212a41c385c13e1109372e8b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26099734212a41c385c13e1109372e8b" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 33569 } }
I20260924 13:47:31.827963 19365 sys_catalog.cc:455] T 00000000000000000000000000000000 P 26099734212a41c385c13e1109372e8b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "26099734212a41c385c13e1109372e8b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26099734212a41c385c13e1109372e8b" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 33569 } } }
I20260924 13:47:31.828074 19365 sys_catalog.cc:458] T 00000000000000000000000000000000 P 26099734212a41c385c13e1109372e8b [sys.catalog]: This master's current role is: LEADER
I20260924 13:47:31.828321 19366 sys_catalog.cc:455] T 00000000000000000000000000000000 P 26099734212a41c385c13e1109372e8b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 26099734212a41c385c13e1109372e8b. Latest consensus state: current_term: 1 leader_uuid: "26099734212a41c385c13e1109372e8b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26099734212a41c385c13e1109372e8b" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 33569 } } }
I20260924 13:47:31.828599 19373 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 13:47:31.829249 19373 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 13:47:31.829389 19366 sys_catalog.cc:458] T 00000000000000000000000000000000 P 26099734212a41c385c13e1109372e8b [sys.catalog]: This master's current role is: LEADER
I20260924 13:47:31.830857 19373 catalog_manager.cc:1347] Generated new cluster ID: 95fbbf9bf6684412ba2b3bbc3b95cbd3
I20260924 13:47:31.830921 19373 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 13:47:31.843556 19373 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 13:47:31.844163 19373 catalog_manager.cc:1504] Loading token signing keys...
I20260924 13:47:31.851986 19373 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 26099734212a41c385c13e1109372e8b: Generated new TSK 0
I20260924 13:47:31.852217 19373 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260924 13:47:31.873850 19363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:31.869105 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55402 (local address 127.13.149.254:33569)
0924 13:47:31.869167 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0924 13:47:31.869170 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 13:47:31.869268 (+ 98us) server_negotiation.cc:407] Connection header received
0924 13:47:31.869350 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:47:31.869352 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:47:31.869366 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:47:31.869467 (+ 101us) server_negotiation.cc:226] Negotiated authn=JWT
0924 13:47:31.869662 (+ 195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:31.870524 (+ 862us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:31.871514 (+ 990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:31.871887 (+ 373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:31.871959 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:47:31.872351 (+ 392us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0924 13:47:31.873615 (+ 1264us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40595/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0924 13:47:31.873757 (+ 142us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:55402: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40595/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":29}
W20260924 13:47:31.873929 19363 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:55402: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40595/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260924 13:47:31.874383 19388 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 13:47:31.869033 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:33569 (local address 127.0.0.1:55402)
0924 13:47:31.869196 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0924 13:47:31.869208 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0924 13:47:31.869273 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:47:31.869501 (+ 228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:47:31.869503 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:47:31.869506 (+ 3us) client_negotiation.cc:189] Negotiated authn=JWT
0924 13:47:31.869576 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:47:31.869578 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:47:31.870674 (+ 1096us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:47:31.870676 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:47:31.871391 (+ 715us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:47:31.871395 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:47:31.872006 (+ 611us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:47:31.872008 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:47:31.872117 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:47:31.872153 (+ 36us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0924 13:47:31.874152 (+ 1999us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40595/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0924 13:47:31.874267 (+ 115us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.13.149.254:33569: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40595/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":83,"threads_started":1}
I20260924 13:47:31.875476 13911 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJN9WRF/build/release/bin/kudu with pid 19291
E20260924 13:47:31.888080 19389 webserver.cc:533] Webserver: sslize failed:
E20260924 13:47:31.888176 19389 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
2026-09-24T13:47:31Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (318 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-09-24T13:47:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T13:47:31Z Disabled control of system clock
I20260924 13:47:31.922907 13911 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJN9WRF/build/release/bin/kudu
/tmp/dist-test-taskJN9WRF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790257629041645-13911-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790257629041645-13911-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790257629041645-13911-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790257629041645-13911-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.149.254:37295
--webserver_interface=127.13.149.254
--webserver_port=0
--builtin_ntp_servers=127.13.149.212:38925
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.149.254:37295
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790257629041645-13911-0/testchainca.cert with env {}
W20260924 13:47:32.005800 19393 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 13:47:32.006032 19393 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 13:47:32.006055 19393 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 13:47:32.007302 19393 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 13:47:32.007346 19393 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 13:47:32.007357 19393 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 13:47:32.007369 19393 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 13:47:32.007380 19393 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260924 13:47:32.007391 19393 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260924 13:47:32.007401 19393 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260924 13:47:32.009013 19393 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.149.212:38925
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790257629041645-13911-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790257629041645-13911-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.149.254:37295
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.149.254:37295
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790257629041645-13911-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.149.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790257629041645-13911-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19393
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790257629041645-13911-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 12:29:37 UTC on ccycloud.quasar-iqzbxn.root.comops.site
build id 4670106
I20260924 13:47:32.009246 19393 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 13:47:32.009421 19393 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 13:47:32.014433 19398 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 13:47:32.014587 19393 server_base.cc:1034] running on GCE node
W20260924 13:47:32.014433 19401 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 13:47:32.014468 19399 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 13:47:32.014956 19393 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 13:47:32.015273 19393 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 13:47:32.016438 19393 hybrid_clock.cc:648] HybridClock initialized: now 1790257652016421 us; error 38 us; skew 500 ppm
I20260924 13:47:32.017855 19393 webserver.cc:460] Webserver started at http://127.13.149.254:46245/ using document root <none> and password file <none>
I20260924 13:47:32.018113 19393 fs_manager.cc:359] Metadata directory not provided
I20260924 13:47:32.018188 19393 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 13:47:32.018321 19393 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 13:47:32.019402 19393 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790257629041645-13911-0/minicluster-data/master-0/data/instance:
uuid: "9d2402aea35c499588f8188b52b36ed1"
format_stamp: "Formatted at 2026-09-24 13:47:32 on dist-test-slave-sjl2"
server_key: "638c5f94448606d70b98324a7acfc88c"
server_key_iv: "828779bef4e4b510cc4b648a137f0b38"
server_key_version: "encryptionkey@0"
I20260924 13:47:32.019753 19393 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790257629041645-13911-0/minicluster-data/master-0/wal/instance:
uuid: "9d2402aea35c499588f8188b52b36ed1"
format_stamp: "Formatted at 2026-09-24 13:47:32 on dist-test-slave-sjl2"
server_key: "638c5f94448606d70b98324a7acfc88c"
server_key_iv: "828779bef4e4b510cc4b648a137f0b38"
server_key_version: "encryptionkey@0"
I20260924 13:47:32.021068 19393 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260924 13:47:32.021807 19407 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 13:47:32.021952 19393 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 13:47:32.022032 19393 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790257629041645-13911-0/minicluster-data/master-0/data,/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790257629041645-13911-0/minicluster-data/master-0/wal
uuid: "9d2402aea35c499588f8188b52b36ed1"
format_stamp: "Formatted at 2026-09-24 13:47:32 on dist-test-slave-sjl2"
server_key: "638c5f94448606d70b98324a7acfc88c"
server_key_iv: "828779bef4e4b510cc4b648a137f0b38"
server_key_version: "encryptionkey@0"
I20260924 13:47:32.022104 19393 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790257629041645-13911-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790257629041645-13911-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790257629041645-13911-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 13:47:32.036377 19393 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 13:47:32.036715 19393 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 13:47:32.036840 19393 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 13:47:32.041074 19393 rpc_server.cc:307] RPC server started. Bound to: 127.13.149.254:37295
I20260924 13:47:32.041129 19460 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.149.254:37295 every 8 connection(s)
I20260924 13:47:32.041471 19393 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790257629041645-13911-0/minicluster-data/master-0/data/info.pb
I20260924 13:47:32.042430 19461 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 13:47:32.044873 19461 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9d2402aea35c499588f8188b52b36ed1: Bootstrap starting.
I20260924 13:47:32.045568 19461 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9d2402aea35c499588f8188b52b36ed1: Neither blocks nor log segments found. Creating new log.
I20260924 13:47:32.045893 19461 log.cc:824] T 00000000000000000000000000000000 P 9d2402aea35c499588f8188b52b36ed1: Log is configured to *not* fsync() on all Append() calls
I20260924 13:47:32.046757 19461 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9d2402aea35c499588f8188b52b36ed1: No bootstrap required, opened a new log
I20260924 13:47:32.047889 13911 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJN9WRF/build/release/bin/kudu as pid 19393
I20260924 13:47:32.047984 13911 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790257629041645-13911-0/minicluster-data/master-0/wal/instance
I20260924 13:47:32.048162 13911 external_mini_cluster.cc:1428] Setting key 49a675be6eac2cfd21b2186050e5e2a6
I20260924 13:47:32.048666 19461 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9d2402aea35c499588f8188b52b36ed1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d2402aea35c499588f8188b52b36ed1" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 37295 } }
I20260924 13:47:32.048870 19461 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9d2402aea35c499588f8188b52b36ed1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 13:47:32.048909 19461 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9d2402aea35c499588f8188b52b36ed1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9d2402aea35c499588f8188b52b36ed1, State: Initialized, Role: FOLLOWER
I20260924 13:47:32.049067 19461 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9d2402aea35c499588f8188b52b36ed1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d2402aea35c499588f8188b52b36ed1" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 37295 } }
I20260924 13:47:32.049134 19461 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9d2402aea35c499588f8188b52b36ed1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 13:47:32.049165 19461 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9d2402aea35c499588f8188b52b36ed1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 13:47:32.049213 19461 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9d2402aea35c499588f8188b52b36ed1 [term 0 FOLLOWER]: Advancing to term 1
I20260924 13:47:32.053615 19465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:32.048752 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35156 (local address 127.13.149.254:37295)
0924 13:47:32.049130 (+ 378us) server_negotiation.cc:206] Beginning negotiation
0924 13:47:32.049141 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0924 13:47:32.049176 (+ 35us) server_negotiation.cc:407] Connection header received
0924 13:47:32.049300 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:47:32.049307 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:47:32.049369 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:47:32.049493 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:47:32.049741 (+ 248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:32.050908 (+ 1167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:32.051670 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:32.052133 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:32.052253 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:47:32.052940 (+ 687us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:47:32.052946 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:47:32.052946 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 13:47:32.053145 (+ 199us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:47:32.053181 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 13:47:32.053195 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:47:32.053252 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0924 13:47:32.053341 (+ 89us) server_negotiation.cc:299] Negotiation successful
0924 13:47:32.053434 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":280,"thread_start_us":102,"threads_started":1}
I20260924 13:47:32.054659 13911 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260924 13:47:32.125491 19461 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9d2402aea35c499588f8188b52b36ed1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d2402aea35c499588f8188b52b36ed1" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 37295 } }
I20260924 13:47:32.125690 19461 leader_election.cc:302] T 00000000000000000000000000000000 P 9d2402aea35c499588f8188b52b36ed1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9d2402aea35c499588f8188b52b36ed1; no voters:
I20260924 13:47:32.125933 19461 leader_election.cc:288] T 00000000000000000000000000000000 P 9d2402aea35c499588f8188b52b36ed1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 13:47:32.126008 19466 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9d2402aea35c499588f8188b52b36ed1 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 13:47:32.126194 19466 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9d2402aea35c499588f8188b52b36ed1 [term 1 LEADER]: Becoming Leader. State: Replica: 9d2402aea35c499588f8188b52b36ed1, State: Running, Role: LEADER
I20260924 13:47:32.126245 19461 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9d2402aea35c499588f8188b52b36ed1 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 13:47:32.126302 19466 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9d2402aea35c499588f8188b52b36ed1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d2402aea35c499588f8188b52b36ed1" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 37295 } }
I20260924 13:47:32.126719 19467 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9d2402aea35c499588f8188b52b36ed1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9d2402aea35c499588f8188b52b36ed1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d2402aea35c499588f8188b52b36ed1" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 37295 } } }
I20260924 13:47:32.126807 19467 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9d2402aea35c499588f8188b52b36ed1 [sys.catalog]: This master's current role is: LEADER
I20260924 13:47:32.127010 19468 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9d2402aea35c499588f8188b52b36ed1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9d2402aea35c499588f8188b52b36ed1. Latest consensus state: current_term: 1 leader_uuid: "9d2402aea35c499588f8188b52b36ed1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d2402aea35c499588f8188b52b36ed1" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 37295 } } }
I20260924 13:47:32.127117 19468 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9d2402aea35c499588f8188b52b36ed1 [sys.catalog]: This master's current role is: LEADER
I20260924 13:47:32.127437 19474 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 13:47:32.127925 19474 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 13:47:32.130244 19474 catalog_manager.cc:1347] Generated new cluster ID: 6f8ad5df0dc64e50935774fb75506b54
I20260924 13:47:32.130306 19474 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 13:47:32.147331 19474 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 13:47:32.147888 19474 catalog_manager.cc:1504] Loading token signing keys...
I20260924 13:47:32.150795 19474 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9d2402aea35c499588f8188b52b36ed1: Generated new TSK 0
I20260924 13:47:32.150987 19474 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260924 13:47:32.227353 19465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:32.158712 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35170 (local address 127.13.149.254:37295)
0924 13:47:32.158759 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0924 13:47:32.158762 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 13:47:32.158859 (+ 97us) server_negotiation.cc:407] Connection header received
0924 13:47:32.158950 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:47:32.158951 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:47:32.158961 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:47:32.159019 (+ 58us) server_negotiation.cc:226] Negotiated authn=JWT
0924 13:47:32.159154 (+ 135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:32.159650 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:32.160850 (+ 1200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:32.161102 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:32.161180 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:47:32.161416 (+ 236us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0924 13:47:32.227130 (+ 65714us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0924 13:47:32.227290 (+ 160us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:35170: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":24}
W20260924 13:47:32.227417 19465 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:35170: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260924 13:47:32.227560 19490 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 13:47:32.158626 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:37295 (local address 127.0.0.1:35170)
0924 13:47:32.158786 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0924 13:47:32.158797 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0924 13:47:32.158871 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:47:32.159037 (+ 166us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:47:32.159038 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:47:32.159040 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0924 13:47:32.159102 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:47:32.159104 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:47:32.159726 (+ 622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:47:32.159727 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:47:32.160708 (+ 981us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:47:32.160713 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:47:32.161205 (+ 492us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:47:32.161206 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:47:32.161274 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:47:32.161281 (+ 7us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0924 13:47:32.227291 (+ 66010us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0924 13:47:32.227453 (+ 162us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.13.149.254:37295: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":83,"threads_started":1}
I20260924 13:47:32.227957 13911 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJN9WRF/build/release/bin/kudu with pid 19393
2026-09-24T13:47:32Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (330 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (92 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (88 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790257629041645-13911-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 13:47:33 dist-test-slave-sjl2 krb5kdc[19502](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 13:47:33 dist-test-slave-sjl2 krb5kdc[19502](info): set up 2 sockets
Sep 24 13:47:33 dist-test-slave-sjl2 krb5kdc[19502](info): commencing operation
krb5kdc: starting...
W20260924 13:47:34.206055 13911 mini_kdc.cc:121] Time spent starting KDC: real 1.780s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790257629041645-13911-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790257629041645-13911-0/krb5kdc/test-user.keytab.
Sep 24 13:47:34 dist-test-slave-sjl2 krb5kdc[19502](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257654, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T13:47:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T13:47:34Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.149.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.149.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.149.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.254.keytab.
Entry for principal kudu/127.13.149.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.254.keytab.
I20260924 13:47:34.332840 13911 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJN9WRF/build/release/bin/kudu
/tmp/dist-test-taskJN9WRF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790257629041645-13911-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790257629041645-13911-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790257629041645-13911-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790257629041645-13911-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.149.254:33469
--webserver_interface=127.13.149.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.254.keytab
--principal=kudu/127.13.149.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.149.212:39287
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.149.254:33469
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790257629041645-13911-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790257629041645-13911-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790257629041645-13911-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 13:47:34.414111 19516 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 13:47:34.414363 19516 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 13:47:34.414387 19516 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 13:47:34.415652 19516 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 13:47:34.415699 19516 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 13:47:34.415714 19516 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 13:47:34.415724 19516 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 13:47:34.415735 19516 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 13:47:34.417351 19516 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.149.212:39287
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790257629041645-13911-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790257629041645-13911-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.149.254:33469
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.254.keytab
--principal=kudu/127.13.149.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.149.254:33469
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790257629041645-13911-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.149.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19516
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790257629041645-13911-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 12:29:37 UTC on ccycloud.quasar-iqzbxn.root.comops.site
build id 4670106
I20260924 13:47:34.417577 19516 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 13:47:34.417771 19516 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 13:47:34.422851 19521 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 13:47:34.422873 19522 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 13:47:34.422851 19524 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 13:47:34.422928 19516 server_base.cc:1034] running on GCE node
I20260924 13:47:34.423350 19516 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 13:47:34.423656 19516 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 13:47:34.424816 19516 hybrid_clock.cc:648] HybridClock initialized: now 1790257654424803 us; error 36 us; skew 500 ppm
Sep 24 13:47:34 dist-test-slave-sjl2 krb5kdc[19502](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257654, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.149.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 13:47:34.427481 19516 init.cc:377] Logged in from keytab as kudu/127.13.149.254@KRBTEST.COM (short username kudu)
I20260924 13:47:34.428017 19516 webserver.cc:460] Webserver started at http://127.13.149.254:38709/ using document root <none> and password file <none>
I20260924 13:47:34.428236 19516 fs_manager.cc:359] Metadata directory not provided
I20260924 13:47:34.428284 19516 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 13:47:34.428414 19516 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 13:47:34.429569 19516 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790257629041645-13911-0/minicluster-data/master-0/data/instance:
uuid: "387af6429ec14909aee9bd04a6b6b3b2"
format_stamp: "Formatted at 2026-09-24 13:47:34 on dist-test-slave-sjl2"
server_key: "85fe2032bd7f88565c6d067bd78da4ae"
server_key_iv: "8cb3435ed8278c521971d693bcce677d"
server_key_version: "encryptionkey@0"
I20260924 13:47:34.429955 19516 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790257629041645-13911-0/minicluster-data/master-0/wal/instance:
uuid: "387af6429ec14909aee9bd04a6b6b3b2"
format_stamp: "Formatted at 2026-09-24 13:47:34 on dist-test-slave-sjl2"
server_key: "85fe2032bd7f88565c6d067bd78da4ae"
server_key_iv: "8cb3435ed8278c521971d693bcce677d"
server_key_version: "encryptionkey@0"
I20260924 13:47:34.431299 19516 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260924 13:47:34.432080 19531 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 13:47:34.432247 19516 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 13:47:34.432341 19516 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790257629041645-13911-0/minicluster-data/master-0/data,/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790257629041645-13911-0/minicluster-data/master-0/wal
uuid: "387af6429ec14909aee9bd04a6b6b3b2"
format_stamp: "Formatted at 2026-09-24 13:47:34 on dist-test-slave-sjl2"
server_key: "85fe2032bd7f88565c6d067bd78da4ae"
server_key_iv: "8cb3435ed8278c521971d693bcce677d"
server_key_version: "encryptionkey@0"
I20260924 13:47:34.432411 19516 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790257629041645-13911-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790257629041645-13911-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790257629041645-13911-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 13:47:34.447309 19516 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 13:47:34.449294 19516 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 13:47:34.449443 19516 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 13:47:34.453401 19516 rpc_server.cc:307] RPC server started. Bound to: 127.13.149.254:33469
I20260924 13:47:34.453441 19583 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.149.254:33469 every 8 connection(s)
I20260924 13:47:34.453786 19516 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790257629041645-13911-0/minicluster-data/master-0/data/info.pb
I20260924 13:47:34.454622 19584 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 13:47:34.457012 19584 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 387af6429ec14909aee9bd04a6b6b3b2: Bootstrap starting.
I20260924 13:47:34.457629 19584 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 387af6429ec14909aee9bd04a6b6b3b2: Neither blocks nor log segments found. Creating new log.
I20260924 13:47:34.457765 13911 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJN9WRF/build/release/bin/kudu as pid 19516
I20260924 13:47:34.457870 13911 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790257629041645-13911-0/minicluster-data/master-0/wal/instance
I20260924 13:47:34.457929 19584 log.cc:824] T 00000000000000000000000000000000 P 387af6429ec14909aee9bd04a6b6b3b2: Log is configured to *not* fsync() on all Append() calls
I20260924 13:47:34.458065 13911 external_mini_cluster.cc:1428] Setting key afd40a189755a27c76472c51fda78e84
I20260924 13:47:34.458907 19584 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 387af6429ec14909aee9bd04a6b6b3b2: No bootstrap required, opened a new log
I20260924 13:47:34.461127 19584 raft_consensus.cc:348] T 00000000000000000000000000000000 P 387af6429ec14909aee9bd04a6b6b3b2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "387af6429ec14909aee9bd04a6b6b3b2" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 33469 } }
I20260924 13:47:34.461319 19584 raft_consensus.cc:374] T 00000000000000000000000000000000 P 387af6429ec14909aee9bd04a6b6b3b2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 13:47:34.461357 19584 raft_consensus.cc:729] T 00000000000000000000000000000000 P 387af6429ec14909aee9bd04a6b6b3b2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 387af6429ec14909aee9bd04a6b6b3b2, State: Initialized, Role: FOLLOWER
I20260924 13:47:34.461503 19584 consensus_queue.cc:260] T 00000000000000000000000000000000 P 387af6429ec14909aee9bd04a6b6b3b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "387af6429ec14909aee9bd04a6b6b3b2" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 33469 } }
I20260924 13:47:34.461585 19584 raft_consensus.cc:388] T 00000000000000000000000000000000 P 387af6429ec14909aee9bd04a6b6b3b2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 13:47:34.461618 19584 raft_consensus.cc:482] T 00000000000000000000000000000000 P 387af6429ec14909aee9bd04a6b6b3b2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 13:47:34.461668 19584 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 387af6429ec14909aee9bd04a6b6b3b2 [term 0 FOLLOWER]: Advancing to term 1
I20260924 13:47:34.462301 19584 raft_consensus.cc:504] T 00000000000000000000000000000000 P 387af6429ec14909aee9bd04a6b6b3b2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "387af6429ec14909aee9bd04a6b6b3b2" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 33469 } }
I20260924 13:47:34.462404 19584 leader_election.cc:302] T 00000000000000000000000000000000 P 387af6429ec14909aee9bd04a6b6b3b2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 387af6429ec14909aee9bd04a6b6b3b2; no voters:
I20260924 13:47:34.462594 19584 leader_election.cc:288] T 00000000000000000000000000000000 P 387af6429ec14909aee9bd04a6b6b3b2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 13:47:34.462638 19589 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 387af6429ec14909aee9bd04a6b6b3b2 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 13:47:34.462791 19589 raft_consensus.cc:686] T 00000000000000000000000000000000 P 387af6429ec14909aee9bd04a6b6b3b2 [term 1 LEADER]: Becoming Leader. State: Replica: 387af6429ec14909aee9bd04a6b6b3b2, State: Running, Role: LEADER
I20260924 13:47:34.462915 19584 sys_catalog.cc:564] T 00000000000000000000000000000000 P 387af6429ec14909aee9bd04a6b6b3b2 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 13:47:34.462927 19589 consensus_queue.cc:237] T 00000000000000000000000000000000 P 387af6429ec14909aee9bd04a6b6b3b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "387af6429ec14909aee9bd04a6b6b3b2" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 33469 } }
I20260924 13:47:34.463389 19591 sys_catalog.cc:455] T 00000000000000000000000000000000 P 387af6429ec14909aee9bd04a6b6b3b2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 387af6429ec14909aee9bd04a6b6b3b2. Latest consensus state: current_term: 1 leader_uuid: "387af6429ec14909aee9bd04a6b6b3b2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "387af6429ec14909aee9bd04a6b6b3b2" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 33469 } } }
I20260924 13:47:34.463404 19590 sys_catalog.cc:455] T 00000000000000000000000000000000 P 387af6429ec14909aee9bd04a6b6b3b2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "387af6429ec14909aee9bd04a6b6b3b2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "387af6429ec14909aee9bd04a6b6b3b2" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 33469 } } }
I20260924 13:47:34.463555 19591 sys_catalog.cc:458] T 00000000000000000000000000000000 P 387af6429ec14909aee9bd04a6b6b3b2 [sys.catalog]: This master's current role is: LEADER
I20260924 13:47:34.463565 19590 sys_catalog.cc:458] T 00000000000000000000000000000000 P 387af6429ec14909aee9bd04a6b6b3b2 [sys.catalog]: This master's current role is: LEADER
I20260924 13:47:34.463815 19598 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 13:47:34.464357 19598 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Sep 24 13:47:34 dist-test-slave-sjl2 krb5kdc[19502](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257654, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.149.254@KRBTEST.COM
I20260924 13:47:34.466038 19598 catalog_manager.cc:1347] Generated new cluster ID: 790ef50ad4a3424ebe408ccde2ac21ce
I20260924 13:47:34.466096 19598 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 13:47:34.469574 19598 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 13:47:34.470124 19598 catalog_manager.cc:1504] Loading token signing keys...
I20260924 13:47:34.472790 19588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:34.458595 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50428 (local address 127.13.149.254:33469)
0924 13:47:34.458794 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0924 13:47:34.458804 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0924 13:47:34.458833 (+ 29us) server_negotiation.cc:407] Connection header received
0924 13:47:34.459326 (+ 493us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:47:34.459333 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:47:34.459391 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:47:34.459521 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:47:34.460167 (+ 646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:34.461163 (+ 996us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:34.462029 (+ 866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:34.462504 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:34.462612 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:47:34.465390 (+ 2778us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:47:34.465395 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:47:34.465395 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 13:47:34.465409 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:47:34.470988 (+ 5579us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:47:34.471459 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:47:34.471463 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:47:34.471463 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:47:34.471592 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:47:34.471784 (+ 192us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:47:34.471784 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:47:34.471785 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:47:34.472105 (+ 320us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:47:34.472313 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0924 13:47:34.472391 (+ 78us) server_negotiation.cc:299] Negotiation successful
0924 13:47:34.472550 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":140,"thread_start_us":89,"threads_started":1}
I20260924 13:47:34.488644 19598 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 387af6429ec14909aee9bd04a6b6b3b2: Generated new TSK 0
I20260924 13:47:34.488983 19598 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.13.149.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.149.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.149.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.193.keytab.
Entry for principal kudu/127.13.149.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.193.keytab.
I20260924 13:47:34.524080 13911 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJN9WRF/build/release/bin/kudu
/tmp/dist-test-taskJN9WRF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790257629041645-13911-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790257629041645-13911-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790257629041645-13911-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790257629041645-13911-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.149.193:0
--local_ip_for_outbound_sockets=127.13.149.193
--webserver_interface=127.13.149.193
--webserver_port=0
--tserver_master_addrs=127.13.149.254:33469
--keytab_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.193.keytab
--principal=kudu/127.13.149.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.149.212:39287
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790257629041645-13911-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790257629041645-13911-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790257629041645-13911-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 13:47:34.607800 19610 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 13:47:34.608048 19610 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 13:47:34.608086 19610 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 13:47:34.609469 19610 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 13:47:34.609529 19610 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 13:47:34.609571 19610 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.149.193
I20260924 13:47:34.611233 19610 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.149.212:39287
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790257629041645-13911-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790257629041645-13911-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.193.keytab
--principal=kudu/127.13.149.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.149.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790257629041645-13911-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.149.193
--webserver_port=0
--tserver_master_addrs=127.13.149.254:33469
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19610
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.149.193
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790257629041645-13911-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 12:29:37 UTC on ccycloud.quasar-iqzbxn.root.comops.site
build id 4670106
I20260924 13:47:34.611506 19610 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 13:47:34.611732 19610 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 13:47:34.617197 19616 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 13:47:34.617183 19615 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 13:47:34.617254 19610 server_base.cc:1034] running on GCE node
W20260924 13:47:34.617183 19618 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 13:47:34.617620 19610 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 13:47:34.617894 19610 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 13:47:34.619067 19610 hybrid_clock.cc:648] HybridClock initialized: now 1790257654619046 us; error 36 us; skew 500 ppm
Sep 24 13:47:34 dist-test-slave-sjl2 krb5kdc[19502](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257654, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.149.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 13:47:34.621642 19610 init.cc:377] Logged in from keytab as kudu/127.13.149.193@KRBTEST.COM (short username kudu)
I20260924 13:47:34.622102 19610 webserver.cc:460] Webserver started at http://127.13.149.193:34857/ using document root <none> and password file <none>
I20260924 13:47:34.622308 19610 fs_manager.cc:359] Metadata directory not provided
I20260924 13:47:34.622355 19610 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 13:47:34.622485 19610 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 13:47:34.623565 19610 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790257629041645-13911-0/minicluster-data/ts-0/data/instance:
uuid: "a58eb1451f3c44bb921e6919513e20cc"
format_stamp: "Formatted at 2026-09-24 13:47:34 on dist-test-slave-sjl2"
server_key: "580de75826aaea666266d7c64d096b14"
server_key_iv: "7294e9a7e2ce1609184095de61848ae8"
server_key_version: "encryptionkey@0"
I20260924 13:47:34.623906 19610 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790257629041645-13911-0/minicluster-data/ts-0/wal/instance:
uuid: "a58eb1451f3c44bb921e6919513e20cc"
format_stamp: "Formatted at 2026-09-24 13:47:34 on dist-test-slave-sjl2"
server_key: "580de75826aaea666266d7c64d096b14"
server_key_iv: "7294e9a7e2ce1609184095de61848ae8"
server_key_version: "encryptionkey@0"
I20260924 13:47:34.625257 19610 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260924 13:47:34.626071 19625 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 13:47:34.626295 19610 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260924 13:47:34.626384 19610 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790257629041645-13911-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790257629041645-13911-0/minicluster-data/ts-0/wal
uuid: "a58eb1451f3c44bb921e6919513e20cc"
format_stamp: "Formatted at 2026-09-24 13:47:34 on dist-test-slave-sjl2"
server_key: "580de75826aaea666266d7c64d096b14"
server_key_iv: "7294e9a7e2ce1609184095de61848ae8"
server_key_version: "encryptionkey@0"
I20260924 13:47:34.626446 19610 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790257629041645-13911-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790257629041645-13911-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790257629041645-13911-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 13:47:34.658475 19610 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 13:47:34.660738 19610 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 13:47:34.660889 19610 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 13:47:34.661152 19610 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 13:47:34.661451 19610 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 13:47:34.661484 19610 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 13:47:34.661514 19610 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 13:47:34.661540 19610 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 13:47:34.667294 19610 rpc_server.cc:307] RPC server started. Bound to: 127.13.149.193:32931
I20260924 13:47:34.667368 19739 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.149.193:32931 every 8 connection(s)
I20260924 13:47:34.667668 19610 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790257629041645-13911-0/minicluster-data/ts-0/data/info.pb
I20260924 13:47:34.669312 13911 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJN9WRF/build/release/bin/kudu as pid 19610
I20260924 13:47:34.669433 13911 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790257629041645-13911-0/minicluster-data/ts-0/wal/instance
I20260924 13:47:34.669603 13911 external_mini_cluster.cc:1428] Setting key 7227cd720c80c04c484cfdec6723413e
Sep 24 13:47:34 dist-test-slave-sjl2 krb5kdc[19502](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257654, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.149.193@KRBTEST.COM for kudu/127.13.149.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.13.149.194@KRBTEST.COM; defaulting to no policy
I20260924 13:47:34.677709 19588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:34.668422 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.149.193:51949 (local address 127.13.149.254:33469)
0924 13:47:34.668466 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0924 13:47:34.668468 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:47:34.668675 (+ 207us) server_negotiation.cc:407] Connection header received
0924 13:47:34.669716 (+ 1041us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:47:34.669718 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:47:34.669732 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:47:34.669804 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:47:34.670560 (+ 756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:34.671056 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:34.672083 (+ 1027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:34.672300 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:34.672359 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:47:34.674723 (+ 2364us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:47:34.674725 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:47:34.674726 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 13:47:34.674733 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:47:34.676026 (+ 1293us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:47:34.676537 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:47:34.676538 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:47:34.676538 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:47:34.676586 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:47:34.676893 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:47:34.676895 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:47:34.676895 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:47:34.677072 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:47:34.677141 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0924 13:47:34.677489 (+ 348us) server_negotiation.cc:299] Negotiation successful
0924 13:47:34.677631 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260924 13:47:34.677748 19742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:34.668383 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:33469 (local address 127.13.149.193:51949)
0924 13:47:34.668552 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0924 13:47:34.668594 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0924 13:47:34.669554 (+ 960us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:47:34.669899 (+ 345us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:47:34.669904 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:47:34.670251 (+ 347us) client_negotiation.cc:189] Negotiated authn=SASL
0924 13:47:34.670461 (+ 210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:47:34.670466 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:47:34.671149 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:47:34.671150 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:47:34.671981 (+ 831us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:47:34.671984 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:47:34.672376 (+ 392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:47:34.672376 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:47:34.672518 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:47:34.673053 (+ 535us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 13:47:34.673062 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0924 13:47:34.674589 (+ 1527us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 13:47:34.676122 (+ 1533us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 13:47:34.676131 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 13:47:34.676135 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0924 13:47:34.676457 (+ 322us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 13:47:34.676644 (+ 187us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 13:47:34.676645 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 13:47:34.676645 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 13:47:34.676787 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 13:47:34.677151 (+ 364us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 13:47:34.677159 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 13:47:34.677325 (+ 166us) client_negotiation.cc:769] Sending connection context
0924 13:47:34.677423 (+ 98us) client_negotiation.cc:240] Negotiation successful
0924 13:47:34.677561 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":86,"threads_started":1}
I20260924 13:47:34.678336 19740 heartbeater.cc:347] Connected to a master server at 127.13.149.254:33469
I20260924 13:47:34.678449 19740 heartbeater.cc:464] Registering TS with master...
I20260924 13:47:34.678678 19740 heartbeater.cc:511] Master 127.13.149.254:33469 requested a full tablet report, sending...
I20260924 13:47:34.679276 19548 ts_manager.cc:194] Registered new tserver with Master: a58eb1451f3c44bb921e6919513e20cc (127.13.149.193:32931)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.149.194@KRBTEST.COM" created.
I20260924 13:47:34.679970 19548 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.149.193@KRBTEST.COM'} at 127.13.149.193:51949
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.149.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.194.keytab.
Entry for principal kudu/127.13.149.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.194.keytab.
I20260924 13:47:34.696905 13911 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJN9WRF/build/release/bin/kudu
/tmp/dist-test-taskJN9WRF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790257629041645-13911-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790257629041645-13911-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790257629041645-13911-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790257629041645-13911-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.149.194:0
--local_ip_for_outbound_sockets=127.13.149.194
--webserver_interface=127.13.149.194
--webserver_port=0
--tserver_master_addrs=127.13.149.254:33469
--keytab_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.194.keytab
--principal=kudu/127.13.149.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.149.212:39287
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790257629041645-13911-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790257629041645-13911-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790257629041645-13911-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 13:47:34.778568 19745 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 13:47:34.778832 19745 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 13:47:34.778859 19745 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 13:47:34.780110 19745 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 13:47:34.780159 19745 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 13:47:34.780185 19745 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.149.194
I20260924 13:47:34.781843 19745 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.149.212:39287
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790257629041645-13911-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790257629041645-13911-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.194.keytab
--principal=kudu/127.13.149.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.149.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790257629041645-13911-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.149.194
--webserver_port=0
--tserver_master_addrs=127.13.149.254:33469
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19745
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.149.194
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790257629041645-13911-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 12:29:37 UTC on ccycloud.quasar-iqzbxn.root.comops.site
build id 4670106
I20260924 13:47:34.782081 19745 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 13:47:34.782284 19745 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 13:47:34.787804 19753 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 13:47:34.787806 19751 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 13:47:34.787942 19745 server_base.cc:1034] running on GCE node
W20260924 13:47:34.787804 19750 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 13:47:34.788271 19745 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 13:47:34.788558 19745 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 13:47:34.789719 19745 hybrid_clock.cc:648] HybridClock initialized: now 1790257654789697 us; error 41 us; skew 500 ppm
Sep 24 13:47:34 dist-test-slave-sjl2 krb5kdc[19502](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257654, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.149.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 13:47:34.792548 19745 init.cc:377] Logged in from keytab as kudu/127.13.149.194@KRBTEST.COM (short username kudu)
I20260924 13:47:34.793087 19745 webserver.cc:460] Webserver started at http://127.13.149.194:41131/ using document root <none> and password file <none>
I20260924 13:47:34.793318 19745 fs_manager.cc:359] Metadata directory not provided
I20260924 13:47:34.793370 19745 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 13:47:34.793499 19745 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 13:47:34.794582 19745 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790257629041645-13911-0/minicluster-data/ts-1/data/instance:
uuid: "56b774958e3d499bb82973662bec572a"
format_stamp: "Formatted at 2026-09-24 13:47:34 on dist-test-slave-sjl2"
server_key: "c992e295508d78a17c92012f7d94ee4a"
server_key_iv: "65b0be55d85bbc5b053b977e83721c87"
server_key_version: "encryptionkey@0"
I20260924 13:47:34.794929 19745 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790257629041645-13911-0/minicluster-data/ts-1/wal/instance:
uuid: "56b774958e3d499bb82973662bec572a"
format_stamp: "Formatted at 2026-09-24 13:47:34 on dist-test-slave-sjl2"
server_key: "c992e295508d78a17c92012f7d94ee4a"
server_key_iv: "65b0be55d85bbc5b053b977e83721c87"
server_key_version: "encryptionkey@0"
I20260924 13:47:34.796272 19745 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260924 13:47:34.797088 19760 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 13:47:34.797255 19745 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260924 13:47:34.797343 19745 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790257629041645-13911-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790257629041645-13911-0/minicluster-data/ts-1/wal
uuid: "56b774958e3d499bb82973662bec572a"
format_stamp: "Formatted at 2026-09-24 13:47:34 on dist-test-slave-sjl2"
server_key: "c992e295508d78a17c92012f7d94ee4a"
server_key_iv: "65b0be55d85bbc5b053b977e83721c87"
server_key_version: "encryptionkey@0"
I20260924 13:47:34.797416 19745 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790257629041645-13911-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790257629041645-13911-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790257629041645-13911-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 13:47:34.814941 19745 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 13:47:34.817001 19745 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 13:47:34.817145 19745 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 13:47:34.817395 19745 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 13:47:34.817691 19745 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 13:47:34.817723 19745 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 13:47:34.817754 19745 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 13:47:34.817768 19745 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 13:47:34.823514 19745 rpc_server.cc:307] RPC server started. Bound to: 127.13.149.194:33871
I20260924 13:47:34.823571 19874 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.149.194:33871 every 8 connection(s)
I20260924 13:47:34.823899 19745 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790257629041645-13911-0/minicluster-data/ts-1/data/info.pb
Sep 24 13:47:34 dist-test-slave-sjl2 krb5kdc[19502](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257654, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.149.194@KRBTEST.COM for kudu/127.13.149.254@KRBTEST.COM
I20260924 13:47:34.832118 13911 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJN9WRF/build/release/bin/kudu as pid 19745
I20260924 13:47:34.832226 13911 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790257629041645-13911-0/minicluster-data/ts-1/wal/instance
I20260924 13:47:34.832414 13911 external_mini_cluster.cc:1428] Setting key e3b8c8bf7aa7528b56b82b0557bec460
I20260924 13:47:34.833526 19588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:34.824676 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.149.194:38841 (local address 127.13.149.254:33469)
0924 13:47:34.824755 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0924 13:47:34.824758 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 13:47:34.824950 (+ 192us) server_negotiation.cc:407] Connection header received
0924 13:47:34.825856 (+ 906us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:47:34.825858 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:47:34.825871 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:47:34.825943 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:47:34.826581 (+ 638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:34.827101 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:34.828137 (+ 1036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:34.828337 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:34.828404 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:47:34.830693 (+ 2289us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:47:34.830695 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:47:34.830695 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 13:47:34.830704 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:47:34.832019 (+ 1315us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:47:34.832530 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:47:34.832531 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:47:34.832532 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:47:34.832598 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:47:34.832838 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:47:34.832839 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:47:34.832840 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:47:34.832999 (+ 159us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:47:34.833060 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0924 13:47:34.833348 (+ 288us) server_negotiation.cc:299] Negotiation successful
0924 13:47:34.833434 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260924 13:47:34.833702 19877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:34.824614 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:33469 (local address 127.13.149.194:38841)
0924 13:47:34.824769 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0924 13:47:34.824813 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0924 13:47:34.825728 (+ 915us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:47:34.826008 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:47:34.826012 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:47:34.826302 (+ 290us) client_negotiation.cc:189] Negotiated authn=SASL
0924 13:47:34.826507 (+ 205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:47:34.826511 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:47:34.827185 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:47:34.827187 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:47:34.828020 (+ 833us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:47:34.828022 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:47:34.828447 (+ 425us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:47:34.828449 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:47:34.828571 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:47:34.829121 (+ 550us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 13:47:34.829130 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0924 13:47:34.830557 (+ 1427us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 13:47:34.832127 (+ 1570us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 13:47:34.832136 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 13:47:34.832138 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 13:47:34.832434 (+ 296us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 13:47:34.832657 (+ 223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 13:47:34.832658 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 13:47:34.832658 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 13:47:34.832784 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 13:47:34.833069 (+ 285us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 13:47:34.833076 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 13:47:34.833214 (+ 138us) client_negotiation.cc:769] Sending connection context
0924 13:47:34.833312 (+ 98us) client_negotiation.cc:240] Negotiation successful
0924 13:47:34.833475 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":70,"threads_started":1}
I20260924 13:47:34.834301 19875 heartbeater.cc:347] Connected to a master server at 127.13.149.254:33469
I20260924 13:47:34.834396 19875 heartbeater.cc:464] Registering TS with master...
I20260924 13:47:34.834594 19875 heartbeater.cc:511] Master 127.13.149.254:33469 requested a full tablet report, sending...
I20260924 13:47:34.834990 19548 ts_manager.cc:194] Registered new tserver with Master: 56b774958e3d499bb82973662bec572a (127.13.149.194:33871)
I20260924 13:47:34.835457 19548 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.149.194@KRBTEST.COM'} at 127.13.149.194:38841
WARNING: no policy specified for kudu/127.13.149.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.149.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.149.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.195.keytab.
Entry for principal kudu/127.13.149.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.195.keytab.
I20260924 13:47:34.859524 13911 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJN9WRF/build/release/bin/kudu
/tmp/dist-test-taskJN9WRF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790257629041645-13911-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790257629041645-13911-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790257629041645-13911-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790257629041645-13911-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.149.195:0
--local_ip_for_outbound_sockets=127.13.149.195
--webserver_interface=127.13.149.195
--webserver_port=0
--tserver_master_addrs=127.13.149.254:33469
--keytab_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.195.keytab
--principal=kudu/127.13.149.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.149.212:39287
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790257629041645-13911-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790257629041645-13911-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790257629041645-13911-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 13:47:34.941220 19880 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 13:47:34.941445 19880 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 13:47:34.941471 19880 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 13:47:34.942737 19880 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 13:47:34.942790 19880 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 13:47:34.942822 19880 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.149.195
I20260924 13:47:34.944473 19880 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.149.212:39287
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790257629041645-13911-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790257629041645-13911-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.195.keytab
--principal=kudu/127.13.149.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.149.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790257629041645-13911-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.149.195
--webserver_port=0
--tserver_master_addrs=127.13.149.254:33469
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19880
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.149.195
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790257629041645-13911-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 12:29:37 UTC on ccycloud.quasar-iqzbxn.root.comops.site
build id 4670106
I20260924 13:47:34.944748 19880 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 13:47:34.944990 19880 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 13:47:34.950080 19885 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 13:47:34.950104 19888 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 13:47:34.950139 19880 server_base.cc:1034] running on GCE node
W20260924 13:47:34.950093 19886 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 13:47:34.950474 19880 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 13:47:34.950701 19880 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 13:47:34.951843 19880 hybrid_clock.cc:648] HybridClock initialized: now 1790257654951823 us; error 29 us; skew 500 ppm
Sep 24 13:47:34 dist-test-slave-sjl2 krb5kdc[19502](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257654, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.149.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 13:47:34.954452 19880 init.cc:377] Logged in from keytab as kudu/127.13.149.195@KRBTEST.COM (short username kudu)
I20260924 13:47:34.954910 19880 webserver.cc:460] Webserver started at http://127.13.149.195:37959/ using document root <none> and password file <none>
I20260924 13:47:34.955116 19880 fs_manager.cc:359] Metadata directory not provided
I20260924 13:47:34.955164 19880 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 13:47:34.955302 19880 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 13:47:34.956377 19880 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790257629041645-13911-0/minicluster-data/ts-2/data/instance:
uuid: "0573e273b69e4079b1a678df0de61d56"
format_stamp: "Formatted at 2026-09-24 13:47:34 on dist-test-slave-sjl2"
server_key: "c4aece7391c44e074a635202f26ec397"
server_key_iv: "7a9c4dc4acdc446f8988e831243ea320"
server_key_version: "encryptionkey@0"
I20260924 13:47:34.956719 19880 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790257629041645-13911-0/minicluster-data/ts-2/wal/instance:
uuid: "0573e273b69e4079b1a678df0de61d56"
format_stamp: "Formatted at 2026-09-24 13:47:34 on dist-test-slave-sjl2"
server_key: "c4aece7391c44e074a635202f26ec397"
server_key_iv: "7a9c4dc4acdc446f8988e831243ea320"
server_key_version: "encryptionkey@0"
I20260924 13:47:34.958135 19880 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260924 13:47:34.959123 19895 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 13:47:34.959308 19880 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260924 13:47:34.959399 19880 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790257629041645-13911-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790257629041645-13911-0/minicluster-data/ts-2/wal
uuid: "0573e273b69e4079b1a678df0de61d56"
format_stamp: "Formatted at 2026-09-24 13:47:34 on dist-test-slave-sjl2"
server_key: "c4aece7391c44e074a635202f26ec397"
server_key_iv: "7a9c4dc4acdc446f8988e831243ea320"
server_key_version: "encryptionkey@0"
I20260924 13:47:34.959477 19880 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790257629041645-13911-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790257629041645-13911-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790257629041645-13911-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 13:47:34.981699 19880 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 13:47:34.984066 19880 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 13:47:34.984218 19880 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 13:47:34.984462 19880 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 13:47:34.984766 19880 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 13:47:34.984797 19880 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 13:47:34.984827 19880 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 13:47:34.984851 19880 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 13:47:34.990859 19880 rpc_server.cc:307] RPC server started. Bound to: 127.13.149.195:34707
I20260924 13:47:34.990906 20008 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.149.195:34707 every 8 connection(s)
I20260924 13:47:34.991231 19880 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790257629041645-13911-0/minicluster-data/ts-2/data/info.pb
I20260924 13:47:34.994611 13911 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJN9WRF/build/release/bin/kudu as pid 19880
I20260924 13:47:34.994716 13911 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790257629041645-13911-0/minicluster-data/ts-2/wal/instance
I20260924 13:47:34.994894 13911 external_mini_cluster.cc:1428] Setting key ee84e459bbee642d60497828d844e9bd
Sep 24 13:47:34 dist-test-slave-sjl2 krb5kdc[19502](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257654, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.149.195@KRBTEST.COM for kudu/127.13.149.254@KRBTEST.COM
I20260924 13:47:35.001180 19588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:34.991946 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.149.195:47197 (local address 127.13.149.254:33469)
0924 13:47:34.992007 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0924 13:47:34.992009 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:47:34.992194 (+ 185us) server_negotiation.cc:407] Connection header received
0924 13:47:34.993148 (+ 954us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:47:34.993149 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:47:34.993161 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:47:34.993225 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:47:34.993864 (+ 639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:34.994377 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:34.995851 (+ 1474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:34.996055 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:34.996112 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:47:34.998280 (+ 2168us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:47:34.998282 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:47:34.998282 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 13:47:34.998290 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:47:34.999730 (+ 1440us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:47:35.000204 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:47:35.000205 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:47:35.000206 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:47:35.000280 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:47:35.000505 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:47:35.000505 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:47:35.000506 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:47:35.000648 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:47:35.000698 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0924 13:47:35.001010 (+ 312us) server_negotiation.cc:299] Negotiation successful
0924 13:47:35.001106 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260924 13:47:35.001340 20011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:34.991904 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:33469 (local address 127.13.149.195:47197)
0924 13:47:34.992054 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0924 13:47:34.992097 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0924 13:47:34.993047 (+ 950us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:47:34.993283 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:47:34.993286 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:47:34.993571 (+ 285us) client_negotiation.cc:189] Negotiated authn=SASL
0924 13:47:34.993784 (+ 213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:47:34.993789 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:47:34.994495 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:47:34.994496 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:47:34.995750 (+ 1254us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:47:34.995753 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:47:34.996123 (+ 370us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:47:34.996124 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:47:34.996241 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:47:34.996759 (+ 518us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 13:47:34.996767 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0924 13:47:34.998180 (+ 1413us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 13:47:34.999829 (+ 1649us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 13:47:34.999838 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 13:47:34.999840 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 13:47:35.000092 (+ 252us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 13:47:35.000342 (+ 250us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 13:47:35.000342 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 13:47:35.000343 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 13:47:35.000457 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 13:47:35.000697 (+ 240us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 13:47:35.000701 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 13:47:35.000831 (+ 130us) client_negotiation.cc:769] Sending connection context
0924 13:47:35.000949 (+ 118us) client_negotiation.cc:240] Negotiation successful
0924 13:47:35.001095 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":78,"threads_started":1}
I20260924 13:47:35.002899 20009 heartbeater.cc:347] Connected to a master server at 127.13.149.254:33469
I20260924 13:47:35.003010 20009 heartbeater.cc:464] Registering TS with master...
I20260924 13:47:35.003253 20009 heartbeater.cc:511] Master 127.13.149.254:33469 requested a full tablet report, sending...
I20260924 13:47:35.003684 19548 ts_manager.cc:194] Registered new tserver with Master: 0573e273b69e4079b1a678df0de61d56 (127.13.149.195:34707)
I20260924 13:47:35.004068 19548 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.149.195@KRBTEST.COM'} at 127.13.149.195:47197
I20260924 13:47:35.006690 13911 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790257629041645-13911-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790257629041645-13911-0/krb5kdc/test-admin.keytab.
Sep 24 13:47:35 dist-test-slave-sjl2 krb5kdc[19502](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257655, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 13:47:35.022542 13911 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260924 13:47:35.024349 20016 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 13:47:35.023481 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35342 (local address 127.13.149.193:32931)
0924 13:47:35.023679 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0924 13:47:35.023684 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 13:47:35.023696 (+ 12us) server_negotiation.cc:407] Connection header received
0924 13:47:35.023722 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:47:35.023723 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:47:35.023739 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:47:35.023811 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:47:35.024270 (+ 459us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35342: BlockingRecv error: recv got EOF from 127.0.0.1:35342 (error 108)
Metrics: {"server-negotiator.queue_time_us":158,"thread_start_us":92,"threads_started":1}
W20260924 13:47:35.024518 13911 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790257629041645-13911-0/krb5kdc/krb5cc' of type FILE
Sep 24 13:47:35 dist-test-slave-sjl2 krb5kdc[19502](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257655, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 13:47:35.025600 13911 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Sep 24 13:47:35 dist-test-slave-sjl2 krb5kdc[19502](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257655, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 13:47:35.026516 13911 init.cc:295] Successfully reacquired a new kerberos TGT
Sep 24 13:47:35 dist-test-slave-sjl2 krb5kdc[19502](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257655, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.149.193@KRBTEST.COM
I20260924 13:47:35.034610 20016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:35.027085 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35346 (local address 127.13.149.193:32931)
0924 13:47:35.027125 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0924 13:47:35.027127 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:47:35.027156 (+ 29us) server_negotiation.cc:407] Connection header received
0924 13:47:35.027222 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:47:35.027223 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:47:35.027233 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:47:35.027279 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:47:35.027702 (+ 423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:35.028443 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:35.029372 (+ 929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:35.029741 (+ 369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:35.029805 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:47:35.032034 (+ 2229us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:47:35.032044 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:47:35.032044 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 13:47:35.032051 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:47:35.033585 (+ 1534us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:47:35.033920 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:47:35.033923 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:47:35.033923 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:47:35.033976 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:47:35.034104 (+ 128us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:47:35.034105 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:47:35.034105 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:47:35.034249 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:47:35.034366 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0924 13:47:35.034410 (+ 44us) server_negotiation.cc:299] Negotiation successful
0924 13:47:35.034497 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
W20260924 13:47:35.036455 20016 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 13:47:35.035975 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35362 (local address 127.13.149.193:32931)
0924 13:47:35.036015 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0924 13:47:35.036017 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:47:35.036078 (+ 61us) server_negotiation.cc:407] Connection header received
0924 13:47:35.036124 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:47:35.036125 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:47:35.036135 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:47:35.036165 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:47:35.036403 (+ 238us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35362: BlockingRecv error: recv got EOF from 127.0.0.1:35362 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260924 13:47:35.036599 13911 init.cc:244] Error while opening credential cache '/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790257629041645-13911-0/krb5kdc/krb5cc' of type FILE: Runtime error: End of credential cache reached (filename: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790257629041645-13911-0/krb5kdc/krb5cc)
Sep 24 13:47:35 dist-test-slave-sjl2 krb5kdc[19502](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257655, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 13:47:35.037659 13911 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Sep 24 13:47:35 dist-test-slave-sjl2 krb5kdc[19502](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257655, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 13:47:35.038414 13911 init.cc:295] Successfully reacquired a new kerberos TGT
Sep 24 13:47:35 dist-test-slave-sjl2 krb5kdc[19502](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257655, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.149.193@KRBTEST.COM
I20260924 13:47:35.045226 20016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:35.038900 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35376 (local address 127.13.149.193:32931)
0924 13:47:35.038954 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0924 13:47:35.038955 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0924 13:47:35.038995 (+ 40us) server_negotiation.cc:407] Connection header received
0924 13:47:35.039039 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:47:35.039040 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:47:35.039048 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:47:35.039089 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:47:35.039416 (+ 327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:35.039907 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:35.040674 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:35.040909 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:35.040998 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:47:35.043062 (+ 2064us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:47:35.043063 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:47:35.043064 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 13:47:35.043072 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:47:35.044166 (+ 1094us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:47:35.044466 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:47:35.044467 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:47:35.044468 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:47:35.044518 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:47:35.044707 (+ 189us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:47:35.044708 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:47:35.044709 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:47:35.044813 (+ 104us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:47:35.044898 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0924 13:47:35.045032 (+ 134us) server_negotiation.cc:299] Negotiation successful
0924 13:47:35.045123 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
W20260924 13:47:35.046732 20016 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 13:47:35.046233 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35388 (local address 127.13.149.193:32931)
0924 13:47:35.046282 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0924 13:47:35.046284 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:47:35.046311 (+ 27us) server_negotiation.cc:407] Connection header received
0924 13:47:35.046372 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:47:35.046373 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:47:35.046382 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:47:35.046426 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:47:35.046684 (+ 258us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35388: BlockingRecv error: recv got EOF from 127.0.0.1:35388 (error 108)
Metrics: {"server-negotiator.queue_time_us":29}
W20260924 13:47:35.046777 13911 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790257629041645-13911-0/krb5kdc/krb5cc' of type FILE
Sep 24 13:47:35 dist-test-slave-sjl2 krb5kdc[19502](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257655, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 13:47:35.047699 13911 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Sep 24 13:47:35 dist-test-slave-sjl2 krb5kdc[19502](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257655, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 13:47:35.048437 13911 init.cc:295] Successfully reacquired a new kerberos TGT
Sep 24 13:47:35 dist-test-slave-sjl2 krb5kdc[19502](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257655, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.149.193@KRBTEST.COM
I20260924 13:47:35.054977 20016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:35.048995 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35392 (local address 127.13.149.193:32931)
0924 13:47:35.049042 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0924 13:47:35.049044 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:47:35.049052 (+ 8us) server_negotiation.cc:407] Connection header received
0924 13:47:35.049117 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:47:35.049118 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:47:35.049127 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:47:35.049171 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:47:35.049504 (+ 333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:35.049982 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:35.050640 (+ 658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:35.050838 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:35.050890 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:47:35.052840 (+ 1950us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:47:35.052841 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:47:35.052842 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 13:47:35.052850 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:47:35.053983 (+ 1133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:47:35.054286 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:47:35.054287 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:47:35.054287 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:47:35.054341 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:47:35.054492 (+ 151us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:47:35.054492 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:47:35.054493 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:47:35.054603 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:47:35.054685 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0924 13:47:35.054803 (+ 118us) server_negotiation.cc:299] Negotiation successful
0924 13:47:35.054881 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260924 13:47:35.055605 13911 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJN9WRF/build/release/bin/kudu with pid 19610
I20260924 13:47:35.060838 13911 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJN9WRF/build/release/bin/kudu with pid 19745
I20260924 13:47:35.067175 13911 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJN9WRF/build/release/bin/kudu with pid 19880
I20260924 13:47:35.071267 13911 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJN9WRF/build/release/bin/kudu with pid 19516
2026-09-24T13:47:35Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (2664 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790257629041645-13911-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 13:47:35 dist-test-slave-sjl2 krb5kdc[20029](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 13:47:35 dist-test-slave-sjl2 krb5kdc[20029](info): set up 2 sockets
Sep 24 13:47:35 dist-test-slave-sjl2 krb5kdc[20029](info): commencing operation
krb5kdc: starting...
W20260924 13:47:36.030130 13911 mini_kdc.cc:121] Time spent starting KDC: real 0.940s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790257629041645-13911-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790257629041645-13911-0/krb5kdc/test-user.keytab.
Sep 24 13:47:36 dist-test-slave-sjl2 krb5kdc[20029](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257656, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T13:47:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T13:47:36Z Disabled control of system clock
WARNING: no policy specified for oryx/127.13.149.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.13.149.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.13.149.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790257629041645-13911-0/krb5kdc/oryx_127.13.149.254.keytab.
Entry for principal oryx/127.13.149.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790257629041645-13911-0/krb5kdc/oryx_127.13.149.254.keytab.
I20260924 13:47:36.147953 13911 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJN9WRF/build/release/bin/kudu
/tmp/dist-test-taskJN9WRF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790257629041645-13911-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790257629041645-13911-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790257629041645-13911-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790257629041645-13911-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.149.254:35859
--webserver_interface=127.13.149.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790257629041645-13911-0/krb5kdc/oryx_127.13.149.254.keytab
--principal=oryx/127.13.149.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.149.212:44199
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.149.254:35859
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790257629041645-13911-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790257629041645-13911-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790257629041645-13911-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 13:47:36.232316 20042 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 13:47:36.232589 20042 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 13:47:36.232620 20042 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 13:47:36.233946 20042 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 13:47:36.234002 20042 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 13:47:36.234015 20042 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 13:47:36.234027 20042 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 13:47:36.234038 20042 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 13:47:36.235775 20042 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.149.212:44199
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790257629041645-13911-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790257629041645-13911-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.149.254:35859
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790257629041645-13911-0/krb5kdc/oryx_127.13.149.254.keytab
--principal=oryx/127.13.149.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.149.254:35859
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790257629041645-13911-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.149.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20042
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790257629041645-13911-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 12:29:37 UTC on ccycloud.quasar-iqzbxn.root.comops.site
build id 4670106
I20260924 13:47:36.236042 20042 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 13:47:36.236299 20042 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 13:47:36.241397 20047 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 13:47:36.241397 20050 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 13:47:36.241573 20042 server_base.cc:1034] running on GCE node
W20260924 13:47:36.241441 20048 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 13:47:36.241863 20042 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 13:47:36.242201 20042 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 13:47:36.243389 20042 hybrid_clock.cc:648] HybridClock initialized: now 1790257656243367 us; error 35 us; skew 500 ppm
Sep 24 13:47:36 dist-test-slave-sjl2 krb5kdc[20029](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257656, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.149.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 13:47:36.251966 20042 init.cc:377] Logged in from keytab as oryx/127.13.149.254@KRBTEST.COM (short username oryx)
I20260924 13:47:36.252501 20042 webserver.cc:460] Webserver started at http://127.13.149.254:37097/ using document root <none> and password file <none>
I20260924 13:47:36.252774 20042 fs_manager.cc:359] Metadata directory not provided
I20260924 13:47:36.252828 20042 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 13:47:36.253000 20042 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 13:47:36.254132 20042 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790257629041645-13911-0/minicluster-data/master-0/data/instance:
uuid: "9976d5567e674e309eecdd83a7ced13a"
format_stamp: "Formatted at 2026-09-24 13:47:36 on dist-test-slave-sjl2"
server_key: "cd4e4e23d445c15e16c048ab57887afe"
server_key_iv: "dbd20bf42d664711ae83af5530f23c3a"
server_key_version: "encryptionkey@0"
I20260924 13:47:36.254483 20042 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790257629041645-13911-0/minicluster-data/master-0/wal/instance:
uuid: "9976d5567e674e309eecdd83a7ced13a"
format_stamp: "Formatted at 2026-09-24 13:47:36 on dist-test-slave-sjl2"
server_key: "cd4e4e23d445c15e16c048ab57887afe"
server_key_iv: "dbd20bf42d664711ae83af5530f23c3a"
server_key_version: "encryptionkey@0"
I20260924 13:47:36.255931 20042 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260924 13:47:36.256793 20057 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 13:47:36.257050 20042 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260924 13:47:36.257144 20042 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790257629041645-13911-0/minicluster-data/master-0/data,/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790257629041645-13911-0/minicluster-data/master-0/wal
uuid: "9976d5567e674e309eecdd83a7ced13a"
format_stamp: "Formatted at 2026-09-24 13:47:36 on dist-test-slave-sjl2"
server_key: "cd4e4e23d445c15e16c048ab57887afe"
server_key_iv: "dbd20bf42d664711ae83af5530f23c3a"
server_key_version: "encryptionkey@0"
I20260924 13:47:36.257225 20042 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790257629041645-13911-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790257629041645-13911-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790257629041645-13911-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 13:47:36.274183 20042 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 13:47:36.276409 20042 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 13:47:36.276548 20042 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 13:47:36.280474 20042 rpc_server.cc:307] RPC server started. Bound to: 127.13.149.254:35859
I20260924 13:47:36.280525 20109 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.149.254:35859 every 8 connection(s)
I20260924 13:47:36.280860 20042 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790257629041645-13911-0/minicluster-data/master-0/data/info.pb
I20260924 13:47:36.281877 20110 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 13:47:36.283275 13911 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJN9WRF/build/release/bin/kudu as pid 20042
I20260924 13:47:36.283396 13911 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790257629041645-13911-0/minicluster-data/master-0/wal/instance
I20260924 13:47:36.283594 13911 external_mini_cluster.cc:1428] Setting key e7646409fe6feb743cea62817da250d4
I20260924 13:47:36.284543 20110 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9976d5567e674e309eecdd83a7ced13a: Bootstrap starting.
I20260924 13:47:36.285210 20110 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9976d5567e674e309eecdd83a7ced13a: Neither blocks nor log segments found. Creating new log.
I20260924 13:47:36.285530 20110 log.cc:824] T 00000000000000000000000000000000 P 9976d5567e674e309eecdd83a7ced13a: Log is configured to *not* fsync() on all Append() calls
I20260924 13:47:36.286185 20110 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9976d5567e674e309eecdd83a7ced13a: No bootstrap required, opened a new log
I20260924 13:47:36.288350 20110 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9976d5567e674e309eecdd83a7ced13a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9976d5567e674e309eecdd83a7ced13a" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 35859 } }
I20260924 13:47:36.288508 20110 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9976d5567e674e309eecdd83a7ced13a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 13:47:36.288542 20110 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9976d5567e674e309eecdd83a7ced13a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9976d5567e674e309eecdd83a7ced13a, State: Initialized, Role: FOLLOWER
I20260924 13:47:36.288699 20110 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9976d5567e674e309eecdd83a7ced13a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9976d5567e674e309eecdd83a7ced13a" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 35859 } }
I20260924 13:47:36.288779 20110 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9976d5567e674e309eecdd83a7ced13a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 13:47:36.288811 20110 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9976d5567e674e309eecdd83a7ced13a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 13:47:36.288861 20110 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9976d5567e674e309eecdd83a7ced13a [term 0 FOLLOWER]: Advancing to term 1
I20260924 13:47:36.289486 20110 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9976d5567e674e309eecdd83a7ced13a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9976d5567e674e309eecdd83a7ced13a" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 35859 } }
I20260924 13:47:36.289598 20110 leader_election.cc:302] T 00000000000000000000000000000000 P 9976d5567e674e309eecdd83a7ced13a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9976d5567e674e309eecdd83a7ced13a; no voters:
I20260924 13:47:36.289789 20110 leader_election.cc:288] T 00000000000000000000000000000000 P 9976d5567e674e309eecdd83a7ced13a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 13:47:36.289867 20115 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9976d5567e674e309eecdd83a7ced13a [term 1 FOLLOWER]: Leader election won for term 1
I20260924 13:47:36.290046 20115 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9976d5567e674e309eecdd83a7ced13a [term 1 LEADER]: Becoming Leader. State: Replica: 9976d5567e674e309eecdd83a7ced13a, State: Running, Role: LEADER
I20260924 13:47:36.290071 20110 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9976d5567e674e309eecdd83a7ced13a [sys.catalog]: configured and running, proceeding with master startup.
Sep 24 13:47:36 dist-test-slave-sjl2 krb5kdc[20029](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257656, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.13.149.254@KRBTEST.COM
I20260924 13:47:36.291329 20115 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9976d5567e674e309eecdd83a7ced13a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9976d5567e674e309eecdd83a7ced13a" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 35859 } }
I20260924 13:47:36.291857 20116 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9976d5567e674e309eecdd83a7ced13a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9976d5567e674e309eecdd83a7ced13a. Latest consensus state: current_term: 1 leader_uuid: "9976d5567e674e309eecdd83a7ced13a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9976d5567e674e309eecdd83a7ced13a" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 35859 } } }
I20260924 13:47:36.292024 20116 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9976d5567e674e309eecdd83a7ced13a [sys.catalog]: This master's current role is: LEADER
I20260924 13:47:36.291872 20117 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9976d5567e674e309eecdd83a7ced13a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9976d5567e674e309eecdd83a7ced13a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9976d5567e674e309eecdd83a7ced13a" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 35859 } } }
I20260924 13:47:36.292392 20117 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9976d5567e674e309eecdd83a7ced13a [sys.catalog]: This master's current role is: LEADER
W20260924 13:47:36.292709 20129 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 9976d5567e674e309eecdd83a7ced13a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260924 13:47:36.292752 20129 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260924 13:47:36.292824 20130 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 13:47:36.293011 20130 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 13:47:36.294942 20113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:36.284156 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34274 (local address 127.13.149.254:35859)
0924 13:47:36.284327 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0924 13:47:36.284335 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0924 13:47:36.284361 (+ 26us) server_negotiation.cc:407] Connection header received
0924 13:47:36.284773 (+ 412us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:47:36.284780 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:47:36.284828 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:47:36.284984 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:47:36.285705 (+ 721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:36.286617 (+ 912us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:36.287629 (+ 1012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:36.288044 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:36.288132 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:47:36.290782 (+ 2650us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:47:36.290787 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:47:36.290787 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 13:47:36.290800 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:47:36.293244 (+ 2444us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:47:36.293699 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:47:36.293702 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:47:36.293703 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:47:36.293786 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:47:36.293998 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:47:36.293999 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:47:36.294000 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:47:36.294308 (+ 308us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:47:36.294456 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0924 13:47:36.294630 (+ 174us) server_negotiation.cc:299] Negotiation successful
0924 13:47:36.294754 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125,"thread_start_us":69,"threads_started":1}
I20260924 13:47:36.295060 20130 catalog_manager.cc:1347] Generated new cluster ID: 1a89448487764376bbc82fc8eb6bb72c
I20260924 13:47:36.295109 20130 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 13:47:36.318132 20130 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 13:47:36.318969 20130 catalog_manager.cc:1504] Loading token signing keys...
I20260924 13:47:36.337926 20130 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9976d5567e674e309eecdd83a7ced13a: Generated new TSK 0
I20260924 13:47:36.338160 20130 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.13.149.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.13.149.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.13.149.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790257629041645-13911-0/krb5kdc/oryx_127.13.149.193.keytab.
Entry for principal oryx/127.13.149.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790257629041645-13911-0/krb5kdc/oryx_127.13.149.193.keytab.
I20260924 13:47:36.364660 13911 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJN9WRF/build/release/bin/kudu
/tmp/dist-test-taskJN9WRF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790257629041645-13911-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790257629041645-13911-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790257629041645-13911-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790257629041645-13911-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.149.193:0
--local_ip_for_outbound_sockets=127.13.149.193
--webserver_interface=127.13.149.193
--webserver_port=0
--tserver_master_addrs=127.13.149.254:35859
--keytab_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790257629041645-13911-0/krb5kdc/oryx_127.13.149.193.keytab
--principal=oryx/127.13.149.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.149.212:44199
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790257629041645-13911-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790257629041645-13911-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790257629041645-13911-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 13:47:36.447340 20136 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 13:47:36.447582 20136 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 13:47:36.447608 20136 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 13:47:36.448789 20136 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 13:47:36.448832 20136 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 13:47:36.448858 20136 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.149.193
I20260924 13:47:36.450443 20136 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.149.212:44199
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790257629041645-13911-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790257629041645-13911-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790257629041645-13911-0/krb5kdc/oryx_127.13.149.193.keytab
--principal=oryx/127.13.149.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.149.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790257629041645-13911-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.149.193
--webserver_port=0
--tserver_master_addrs=127.13.149.254:35859
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20136
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.149.193
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790257629041645-13911-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 12:29:37 UTC on ccycloud.quasar-iqzbxn.root.comops.site
build id 4670106
I20260924 13:47:36.450671 20136 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 13:47:36.450853 20136 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 13:47:36.455785 20144 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 13:47:36.455780 20141 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 13:47:36.455894 20142 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 13:47:36.455935 20136 server_base.cc:1034] running on GCE node
I20260924 13:47:36.456241 20136 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 13:47:36.456467 20136 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 13:47:36.457633 20136 hybrid_clock.cc:648] HybridClock initialized: now 1790257656457616 us; error 27 us; skew 500 ppm
Sep 24 13:47:36 dist-test-slave-sjl2 krb5kdc[20029](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257656, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.149.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 13:47:36.460230 20136 init.cc:377] Logged in from keytab as oryx/127.13.149.193@KRBTEST.COM (short username oryx)
I20260924 13:47:36.460680 20136 webserver.cc:460] Webserver started at http://127.13.149.193:34119/ using document root <none> and password file <none>
I20260924 13:47:36.460879 20136 fs_manager.cc:359] Metadata directory not provided
I20260924 13:47:36.460947 20136 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 13:47:36.461084 20136 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 13:47:36.462157 20136 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790257629041645-13911-0/minicluster-data/ts-0/data/instance:
uuid: "b79e3cc74b9d4c54bd7e3bc4626d08a1"
format_stamp: "Formatted at 2026-09-24 13:47:36 on dist-test-slave-sjl2"
server_key: "ab8f708b7e507f334286734901d5523a"
server_key_iv: "d2e9173d948a1003087b2b6093d702f3"
server_key_version: "encryptionkey@0"
I20260924 13:47:36.462517 20136 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790257629041645-13911-0/minicluster-data/ts-0/wal/instance:
uuid: "b79e3cc74b9d4c54bd7e3bc4626d08a1"
format_stamp: "Formatted at 2026-09-24 13:47:36 on dist-test-slave-sjl2"
server_key: "ab8f708b7e507f334286734901d5523a"
server_key_iv: "d2e9173d948a1003087b2b6093d702f3"
server_key_version: "encryptionkey@0"
I20260924 13:47:36.463965 20136 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260924 13:47:36.465052 20151 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 13:47:36.465253 20136 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260924 13:47:36.465345 20136 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790257629041645-13911-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790257629041645-13911-0/minicluster-data/ts-0/wal
uuid: "b79e3cc74b9d4c54bd7e3bc4626d08a1"
format_stamp: "Formatted at 2026-09-24 13:47:36 on dist-test-slave-sjl2"
server_key: "ab8f708b7e507f334286734901d5523a"
server_key_iv: "d2e9173d948a1003087b2b6093d702f3"
server_key_version: "encryptionkey@0"
I20260924 13:47:36.465422 20136 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790257629041645-13911-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790257629041645-13911-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790257629041645-13911-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 13:47:36.486173 20136 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 13:47:36.488262 20136 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 13:47:36.488385 20136 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 13:47:36.488646 20136 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 13:47:36.488976 20136 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 13:47:36.489017 20136 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 13:47:36.489040 20136 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 13:47:36.489051 20136 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 13:47:36.494894 20136 rpc_server.cc:307] RPC server started. Bound to: 127.13.149.193:44033
I20260924 13:47:36.494943 20264 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.149.193:44033 every 8 connection(s)
I20260924 13:47:36.495257 20136 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790257629041645-13911-0/minicluster-data/ts-0/data/info.pb
I20260924 13:47:36.499929 13911 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJN9WRF/build/release/bin/kudu as pid 20136
I20260924 13:47:36.500047 13911 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790257629041645-13911-0/minicluster-data/ts-0/wal/instance
I20260924 13:47:36.500234 13911 external_mini_cluster.cc:1428] Setting key 81a55aa1547a551968ac59632bff7810
Sep 24 13:47:36 dist-test-slave-sjl2 krb5kdc[20029](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257656, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.149.193@KRBTEST.COM for oryx/127.13.149.254@KRBTEST.COM
I20260924 13:47:36.504887 20113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:36.495974 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.149.193:42693 (local address 127.13.149.254:35859)
0924 13:47:36.496057 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0924 13:47:36.496059 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:47:36.496245 (+ 186us) server_negotiation.cc:407] Connection header received
0924 13:47:36.496752 (+ 507us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:47:36.496753 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:47:36.496762 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:47:36.496811 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:47:36.497803 (+ 992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:36.498306 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:36.499333 (+ 1027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:36.499575 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:36.499636 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:47:36.501841 (+ 2205us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:47:36.501843 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:47:36.501843 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 13:47:36.501852 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:47:36.503229 (+ 1377us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:47:36.503720 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:47:36.503722 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:47:36.503722 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:47:36.503811 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:47:36.504123 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:47:36.504124 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:47:36.504124 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:47:36.504284 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:47:36.504357 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0924 13:47:36.504684 (+ 327us) server_negotiation.cc:299] Negotiation successful
0924 13:47:36.504788 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260924 13:47:36.505010 20267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:36.495939 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:35859 (local address 127.13.149.193:42693)
0924 13:47:36.496121 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0924 13:47:36.496161 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0924 13:47:36.496630 (+ 469us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:47:36.496864 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:47:36.496868 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:47:36.497465 (+ 597us) client_negotiation.cc:189] Negotiated authn=SASL
0924 13:47:36.497683 (+ 218us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:47:36.497688 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:47:36.498398 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:47:36.498399 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:47:36.499240 (+ 841us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:47:36.499242 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:47:36.499641 (+ 399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:47:36.499642 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:47:36.499791 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:47:36.500316 (+ 525us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 13:47:36.500326 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0924 13:47:36.501751 (+ 1425us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 13:47:36.503332 (+ 1581us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 13:47:36.503340 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 13:47:36.503342 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 13:47:36.503618 (+ 276us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 13:47:36.503913 (+ 295us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 13:47:36.503914 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 13:47:36.503914 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 13:47:36.504053 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 13:47:36.504375 (+ 322us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 13:47:36.504383 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 13:47:36.504522 (+ 139us) client_negotiation.cc:769] Sending connection context
0924 13:47:36.504647 (+ 125us) client_negotiation.cc:240] Negotiation successful
0924 13:47:36.504788 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":103,"threads_started":1}
I20260924 13:47:36.505541 20265 heartbeater.cc:347] Connected to a master server at 127.13.149.254:35859
I20260924 13:47:36.505646 20265 heartbeater.cc:464] Registering TS with master...
I20260924 13:47:36.505823 20265 heartbeater.cc:511] Master 127.13.149.254:35859 requested a full tablet report, sending...
I20260924 13:47:36.506407 20074 ts_manager.cc:194] Registered new tserver with Master: b79e3cc74b9d4c54bd7e3bc4626d08a1 (127.13.149.193:44033)
I20260924 13:47:36.507356 20074 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.13.149.193@KRBTEST.COM'} at 127.13.149.193:42693
WARNING: no policy specified for oryx/127.13.149.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.13.149.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.13.149.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790257629041645-13911-0/krb5kdc/oryx_127.13.149.194.keytab.
Entry for principal oryx/127.13.149.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790257629041645-13911-0/krb5kdc/oryx_127.13.149.194.keytab.
I20260924 13:47:36.528169 13911 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJN9WRF/build/release/bin/kudu
/tmp/dist-test-taskJN9WRF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790257629041645-13911-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790257629041645-13911-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790257629041645-13911-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790257629041645-13911-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.149.194:0
--local_ip_for_outbound_sockets=127.13.149.194
--webserver_interface=127.13.149.194
--webserver_port=0
--tserver_master_addrs=127.13.149.254:35859
--keytab_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790257629041645-13911-0/krb5kdc/oryx_127.13.149.194.keytab
--principal=oryx/127.13.149.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.149.212:44199
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790257629041645-13911-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790257629041645-13911-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790257629041645-13911-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 13:47:36.609809 20270 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 13:47:36.610041 20270 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 13:47:36.610075 20270 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 13:47:36.611307 20270 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 13:47:36.611359 20270 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 13:47:36.611395 20270 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.149.194
I20260924 13:47:36.612895 20270 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.149.212:44199
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790257629041645-13911-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790257629041645-13911-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790257629041645-13911-0/krb5kdc/oryx_127.13.149.194.keytab
--principal=oryx/127.13.149.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.149.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790257629041645-13911-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.149.194
--webserver_port=0
--tserver_master_addrs=127.13.149.254:35859
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20270
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.149.194
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790257629041645-13911-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 12:29:37 UTC on ccycloud.quasar-iqzbxn.root.comops.site
build id 4670106
I20260924 13:47:36.613168 20270 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 13:47:36.613402 20270 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260924 13:47:36.618583 20270 server_base.cc:1034] running on GCE node
W20260924 13:47:36.618630 20276 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 13:47:36.618753 20278 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 13:47:36.618602 20275 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 13:47:36.618978 20270 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 13:47:36.619220 20270 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 13:47:36.620376 20270 hybrid_clock.cc:648] HybridClock initialized: now 1790257656620360 us; error 27 us; skew 500 ppm
Sep 24 13:47:36 dist-test-slave-sjl2 krb5kdc[20029](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257656, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.149.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 13:47:36.622924 20270 init.cc:377] Logged in from keytab as oryx/127.13.149.194@KRBTEST.COM (short username oryx)
I20260924 13:47:36.623375 20270 webserver.cc:460] Webserver started at http://127.13.149.194:36543/ using document root <none> and password file <none>
I20260924 13:47:36.623605 20270 fs_manager.cc:359] Metadata directory not provided
I20260924 13:47:36.623653 20270 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 13:47:36.623780 20270 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 13:47:36.624861 20270 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790257629041645-13911-0/minicluster-data/ts-1/data/instance:
uuid: "bc2e044c3d834bebbf3dc17a2c4da709"
format_stamp: "Formatted at 2026-09-24 13:47:36 on dist-test-slave-sjl2"
server_key: "0ad31aa84c904d4f349dcc4b61bd47d0"
server_key_iv: "3721edf7eded99bb41528746a6dbb9e7"
server_key_version: "encryptionkey@0"
I20260924 13:47:36.625264 20270 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790257629041645-13911-0/minicluster-data/ts-1/wal/instance:
uuid: "bc2e044c3d834bebbf3dc17a2c4da709"
format_stamp: "Formatted at 2026-09-24 13:47:36 on dist-test-slave-sjl2"
server_key: "0ad31aa84c904d4f349dcc4b61bd47d0"
server_key_iv: "3721edf7eded99bb41528746a6dbb9e7"
server_key_version: "encryptionkey@0"
I20260924 13:47:36.626705 20270 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260924 13:47:36.627619 20285 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 13:47:36.627846 20270 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260924 13:47:36.627940 20270 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790257629041645-13911-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790257629041645-13911-0/minicluster-data/ts-1/wal
uuid: "bc2e044c3d834bebbf3dc17a2c4da709"
format_stamp: "Formatted at 2026-09-24 13:47:36 on dist-test-slave-sjl2"
server_key: "0ad31aa84c904d4f349dcc4b61bd47d0"
server_key_iv: "3721edf7eded99bb41528746a6dbb9e7"
server_key_version: "encryptionkey@0"
I20260924 13:47:36.628024 20270 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790257629041645-13911-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790257629041645-13911-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790257629041645-13911-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 13:47:36.639765 20270 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 13:47:36.642011 20270 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 13:47:36.642256 20270 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 13:47:36.642501 20270 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 13:47:36.642851 20270 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 13:47:36.642890 20270 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 13:47:36.642921 20270 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 13:47:36.642946 20270 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 13:47:36.648715 20270 rpc_server.cc:307] RPC server started. Bound to: 127.13.149.194:37849
I20260924 13:47:36.648774 20398 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.149.194:37849 every 8 connection(s)
I20260924 13:47:36.649142 20270 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790257629041645-13911-0/minicluster-data/ts-1/data/info.pb
I20260924 13:47:36.653142 13911 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJN9WRF/build/release/bin/kudu as pid 20270
I20260924 13:47:36.653262 13911 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790257629041645-13911-0/minicluster-data/ts-1/wal/instance
I20260924 13:47:36.653460 13911 external_mini_cluster.cc:1428] Setting key 20f9308266ba67651eb7e6614b976dfa
Sep 24 13:47:36 dist-test-slave-sjl2 krb5kdc[20029](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257656, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.149.194@KRBTEST.COM for oryx/127.13.149.254@KRBTEST.COM
I20260924 13:47:36.660281 20113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:36.649942 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.149.194:39481 (local address 127.13.149.254:35859)
0924 13:47:36.650021 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0924 13:47:36.650023 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:47:36.650192 (+ 169us) server_negotiation.cc:407] Connection header received
0924 13:47:36.650768 (+ 576us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:47:36.650769 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:47:36.650782 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:47:36.650842 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:47:36.651832 (+ 990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:36.652338 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:36.653695 (+ 1357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:36.653909 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:36.653972 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:47:36.656762 (+ 2790us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:47:36.656764 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:47:36.656764 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 13:47:36.656773 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:47:36.658252 (+ 1479us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:47:36.658855 (+ 603us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:47:36.658858 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:47:36.658858 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:47:36.658935 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:47:36.659325 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:47:36.659327 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:47:36.659327 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:47:36.659526 (+ 199us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:47:36.659600 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0924 13:47:36.660093 (+ 493us) server_negotiation.cc:299] Negotiation successful
0924 13:47:36.660196 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260924 13:47:36.660540 20401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:36.649870 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:35859 (local address 127.13.149.194:39481)
0924 13:47:36.650058 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0924 13:47:36.650108 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0924 13:47:36.650659 (+ 551us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:47:36.650917 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:47:36.650921 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:47:36.651516 (+ 595us) client_negotiation.cc:189] Negotiated authn=SASL
0924 13:47:36.651729 (+ 213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:47:36.651735 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:47:36.652426 (+ 691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:47:36.652427 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:47:36.653547 (+ 1120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:47:36.653551 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:47:36.653989 (+ 438us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:47:36.653990 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:47:36.654132 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:47:36.654732 (+ 600us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 13:47:36.654742 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0924 13:47:36.656624 (+ 1882us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 13:47:36.658355 (+ 1731us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 13:47:36.658367 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 13:47:36.658371 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0924 13:47:36.658757 (+ 386us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 13:47:36.659000 (+ 243us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 13:47:36.659001 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 13:47:36.659002 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 13:47:36.659192 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 13:47:36.659624 (+ 432us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 13:47:36.659634 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 13:47:36.659929 (+ 295us) client_negotiation.cc:769] Sending connection context
0924 13:47:36.660069 (+ 140us) client_negotiation.cc:240] Negotiation successful
0924 13:47:36.660285 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":84,"threads_started":1}
I20260924 13:47:36.661164 20399 heartbeater.cc:347] Connected to a master server at 127.13.149.254:35859
I20260924 13:47:36.661273 20399 heartbeater.cc:464] Registering TS with master...
I20260924 13:47:36.661470 20399 heartbeater.cc:511] Master 127.13.149.254:35859 requested a full tablet report, sending...
I20260924 13:47:36.661876 20074 ts_manager.cc:194] Registered new tserver with Master: bc2e044c3d834bebbf3dc17a2c4da709 (127.13.149.194:37849)
WARNING: no policy specified for oryx/127.13.149.195@KRBTEST.COM; defaulting to no policy
I20260924 13:47:36.662410 20074 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.13.149.194@KRBTEST.COM'} at 127.13.149.194:39481
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.13.149.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.13.149.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790257629041645-13911-0/krb5kdc/oryx_127.13.149.195.keytab.
Entry for principal oryx/127.13.149.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790257629041645-13911-0/krb5kdc/oryx_127.13.149.195.keytab.
I20260924 13:47:36.680825 13911 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJN9WRF/build/release/bin/kudu
/tmp/dist-test-taskJN9WRF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790257629041645-13911-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790257629041645-13911-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790257629041645-13911-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790257629041645-13911-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.149.195:0
--local_ip_for_outbound_sockets=127.13.149.195
--webserver_interface=127.13.149.195
--webserver_port=0
--tserver_master_addrs=127.13.149.254:35859
--keytab_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790257629041645-13911-0/krb5kdc/oryx_127.13.149.195.keytab
--principal=oryx/127.13.149.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.149.212:44199
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790257629041645-13911-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790257629041645-13911-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790257629041645-13911-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 13:47:36.765081 20404 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 13:47:36.765347 20404 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 13:47:36.765374 20404 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 13:47:36.766616 20404 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 13:47:36.766660 20404 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 13:47:36.766687 20404 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.149.195
I20260924 13:47:36.768216 20404 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.149.212:44199
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790257629041645-13911-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790257629041645-13911-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790257629041645-13911-0/krb5kdc/oryx_127.13.149.195.keytab
--principal=oryx/127.13.149.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.149.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790257629041645-13911-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.149.195
--webserver_port=0
--tserver_master_addrs=127.13.149.254:35859
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20404
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.149.195
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790257629041645-13911-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 12:29:37 UTC on ccycloud.quasar-iqzbxn.root.comops.site
build id 4670106
I20260924 13:47:36.768424 20404 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 13:47:36.768625 20404 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 13:47:36.773558 20412 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 13:47:36.773566 20410 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 13:47:36.773586 20409 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 13:47:36.773954 20404 server_base.cc:1034] running on GCE node
I20260924 13:47:36.774185 20404 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 13:47:36.774399 20404 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 13:47:36.775543 20404 hybrid_clock.cc:648] HybridClock initialized: now 1790257656775527 us; error 28 us; skew 500 ppm
Sep 24 13:47:36 dist-test-slave-sjl2 krb5kdc[20029](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257656, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.149.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 13:47:36.778088 20404 init.cc:377] Logged in from keytab as oryx/127.13.149.195@KRBTEST.COM (short username oryx)
I20260924 13:47:36.778570 20404 webserver.cc:460] Webserver started at http://127.13.149.195:35691/ using document root <none> and password file <none>
I20260924 13:47:36.778753 20404 fs_manager.cc:359] Metadata directory not provided
I20260924 13:47:36.778792 20404 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 13:47:36.778887 20404 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 13:47:36.779909 20404 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790257629041645-13911-0/minicluster-data/ts-2/data/instance:
uuid: "b056fa6235384b0ebb8123a47aaa399e"
format_stamp: "Formatted at 2026-09-24 13:47:36 on dist-test-slave-sjl2"
server_key: "5b0f0937ec2bc45d056a04d8ac78116a"
server_key_iv: "e576006ff804e75656e954ec3c27f446"
server_key_version: "encryptionkey@0"
I20260924 13:47:36.780210 20404 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790257629041645-13911-0/minicluster-data/ts-2/wal/instance:
uuid: "b056fa6235384b0ebb8123a47aaa399e"
format_stamp: "Formatted at 2026-09-24 13:47:36 on dist-test-slave-sjl2"
server_key: "5b0f0937ec2bc45d056a04d8ac78116a"
server_key_iv: "e576006ff804e75656e954ec3c27f446"
server_key_version: "encryptionkey@0"
I20260924 13:47:36.781441 20404 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260924 13:47:36.782115 20419 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 13:47:36.782305 20404 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260924 13:47:36.782400 20404 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790257629041645-13911-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790257629041645-13911-0/minicluster-data/ts-2/wal
uuid: "b056fa6235384b0ebb8123a47aaa399e"
format_stamp: "Formatted at 2026-09-24 13:47:36 on dist-test-slave-sjl2"
server_key: "5b0f0937ec2bc45d056a04d8ac78116a"
server_key_iv: "e576006ff804e75656e954ec3c27f446"
server_key_version: "encryptionkey@0"
I20260924 13:47:36.782459 20404 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790257629041645-13911-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790257629041645-13911-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790257629041645-13911-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 13:47:36.796316 20404 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 13:47:36.798504 20404 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 13:47:36.798630 20404 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 13:47:36.798832 20404 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 13:47:36.799122 20404 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 13:47:36.799158 20404 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 13:47:36.799180 20404 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 13:47:36.799204 20404 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 13:47:36.804711 20404 rpc_server.cc:307] RPC server started. Bound to: 127.13.149.195:41851
I20260924 13:47:36.804760 20532 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.149.195:41851 every 8 connection(s)
I20260924 13:47:36.805131 20404 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790257629041645-13911-0/minicluster-data/ts-2/data/info.pb
I20260924 13:47:36.805805 13911 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJN9WRF/build/release/bin/kudu as pid 20404
I20260924 13:47:36.805908 13911 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790257629041645-13911-0/minicluster-data/ts-2/wal/instance
I20260924 13:47:36.806109 13911 external_mini_cluster.cc:1428] Setting key 7125231dc601ee772f402ef286523b40
Sep 24 13:47:36 dist-test-slave-sjl2 krb5kdc[20029](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257656, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.149.195@KRBTEST.COM for oryx/127.13.149.254@KRBTEST.COM
I20260924 13:47:36.814465 20113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:36.805931 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.149.195:50393 (local address 127.13.149.254:35859)
0924 13:47:36.805982 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0924 13:47:36.805984 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:47:36.806313 (+ 329us) server_negotiation.cc:407] Connection header received
0924 13:47:36.806847 (+ 534us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:47:36.806847 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:47:36.806860 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:47:36.806915 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:47:36.807874 (+ 959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:36.808393 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:36.809512 (+ 1119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:36.809719 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:36.809786 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:47:36.812011 (+ 2225us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:47:36.812012 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:47:36.812012 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 13:47:36.812020 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:47:36.813231 (+ 1211us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:47:36.813583 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:47:36.813584 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:47:36.813584 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:47:36.813661 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:47:36.813857 (+ 196us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:47:36.813857 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:47:36.813857 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:47:36.813963 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:47:36.814011 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0924 13:47:36.814329 (+ 318us) server_negotiation.cc:299] Negotiation successful
0924 13:47:36.814394 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260924 13:47:36.814648 20535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:36.805913 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:35859 (local address 127.13.149.195:50393)
0924 13:47:36.806094 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0924 13:47:36.806153 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0924 13:47:36.806752 (+ 599us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:47:36.806989 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:47:36.806994 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:47:36.807575 (+ 581us) client_negotiation.cc:189] Negotiated authn=SASL
0924 13:47:36.807781 (+ 206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:47:36.807786 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:47:36.808499 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:47:36.808500 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:47:36.809427 (+ 927us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:47:36.809430 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:47:36.809820 (+ 390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:47:36.809821 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:47:36.809943 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:47:36.810503 (+ 560us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 13:47:36.810512 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0924 13:47:36.811893 (+ 1381us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 13:47:36.813306 (+ 1413us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 13:47:36.813312 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 13:47:36.813314 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 13:47:36.813530 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 13:47:36.813711 (+ 181us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 13:47:36.813711 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 13:47:36.813711 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 13:47:36.813809 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 13:47:36.814051 (+ 242us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 13:47:36.814058 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 13:47:36.814195 (+ 137us) client_negotiation.cc:769] Sending connection context
0924 13:47:36.814295 (+ 100us) client_negotiation.cc:240] Negotiation successful
0924 13:47:36.814427 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":95,"threads_started":1}
I20260924 13:47:36.815150 20533 heartbeater.cc:347] Connected to a master server at 127.13.149.254:35859
I20260924 13:47:36.815248 20533 heartbeater.cc:464] Registering TS with master...
I20260924 13:47:36.815428 20533 heartbeater.cc:511] Master 127.13.149.254:35859 requested a full tablet report, sending...
I20260924 13:47:36.815785 20074 ts_manager.cc:194] Registered new tserver with Master: b056fa6235384b0ebb8123a47aaa399e (127.13.149.195:41851)
I20260924 13:47:36.816166 20074 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.13.149.195@KRBTEST.COM'} at 127.13.149.195:50393
I20260924 13:47:36.817994 13911 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.13.149.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.13.149.253@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.13.149.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790257629041645-13911-0/krb5kdc/oryx_127.13.149.253.keytab.
Entry for principal oryx/127.13.149.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790257629041645-13911-0/krb5kdc/oryx_127.13.149.253.keytab.
I20260924 13:47:36.843845 13911 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJN9WRF/build/release/bin/kudu
/tmp/dist-test-taskJN9WRF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790257629041645-13911-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790257629041645-13911-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790257629041645-13911-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790257629041645-13911-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.149.253:38973
--webserver_interface=127.13.149.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790257629041645-13911-0/krb5kdc/oryx_127.13.149.253.keytab
--principal=oryx/127.13.149.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.149.212:44199
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.149.254:35859,127.13.149.253:38973
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790257629041645-13911-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790257629041645-13911-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790257629041645-13911-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 13:47:36.928272 20538 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 13:47:36.928495 20538 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 13:47:36.928527 20538 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 13:47:36.929790 20538 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 13:47:36.929847 20538 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 13:47:36.929865 20538 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 13:47:36.929880 20538 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 13:47:36.929896 20538 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 13:47:36.931793 20538 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.149.212:44199
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790257629041645-13911-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790257629041645-13911-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.13.149.254:35859,127.13.149.253:38973
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790257629041645-13911-0/krb5kdc/oryx_127.13.149.253.keytab
--principal=oryx/127.13.149.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.149.253:38973
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790257629041645-13911-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.13.149.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20538
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790257629041645-13911-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 12:29:37 UTC on ccycloud.quasar-iqzbxn.root.comops.site
build id 4670106
W20260924 13:47:36.931942 20538 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.13.149.254:35859,127.13.149.253:38973'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260924 13:47:36.932099 20538 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 13:47:36.932332 20538 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 13:47:36.937165 20544 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 13:47:36.937180 20546 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 13:47:36.937281 20538 server_base.cc:1034] running on GCE node
W20260924 13:47:36.937187 20543 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 13:47:36.937621 20538 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 13:47:36.937935 20538 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 13:47:36.939079 20538 hybrid_clock.cc:648] HybridClock initialized: now 1790257656939064 us; error 37 us; skew 500 ppm
Sep 24 13:47:36 dist-test-slave-sjl2 krb5kdc[20029](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257656, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.149.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 13:47:36.941525 20538 init.cc:377] Logged in from keytab as oryx/127.13.149.253@KRBTEST.COM (short username oryx)
I20260924 13:47:36.941984 20538 webserver.cc:460] Webserver started at http://127.13.149.253:36981/ using document root <none> and password file <none>
I20260924 13:47:36.942189 20538 fs_manager.cc:359] Metadata directory not provided
I20260924 13:47:36.942230 20538 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 13:47:36.942332 20538 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 13:47:36.943372 20538 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790257629041645-13911-0/minicluster-data/master-1/data/instance:
uuid: "1d4faf110d7a41cc8a2113c5b74a298a"
format_stamp: "Formatted at 2026-09-24 13:47:36 on dist-test-slave-sjl2"
server_key: "a60161f45e778623c5e6b6f536b04d81"
server_key_iv: "338522ce603bcd6a49c50315f1afa730"
server_key_version: "encryptionkey@0"
I20260924 13:47:36.943665 20538 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790257629041645-13911-0/minicluster-data/master-1/wal/instance:
uuid: "1d4faf110d7a41cc8a2113c5b74a298a"
format_stamp: "Formatted at 2026-09-24 13:47:36 on dist-test-slave-sjl2"
server_key: "a60161f45e778623c5e6b6f536b04d81"
server_key_iv: "338522ce603bcd6a49c50315f1afa730"
server_key_version: "encryptionkey@0"
I20260924 13:47:36.944962 20538 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260924 13:47:36.945629 20553 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 13:47:36.945761 20538 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260924 13:47:36.945843 20538 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790257629041645-13911-0/minicluster-data/master-1/data,/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790257629041645-13911-0/minicluster-data/master-1/wal
uuid: "1d4faf110d7a41cc8a2113c5b74a298a"
format_stamp: "Formatted at 2026-09-24 13:47:36 on dist-test-slave-sjl2"
server_key: "a60161f45e778623c5e6b6f536b04d81"
server_key_iv: "338522ce603bcd6a49c50315f1afa730"
server_key_version: "encryptionkey@0"
I20260924 13:47:36.945919 20538 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790257629041645-13911-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790257629041645-13911-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790257629041645-13911-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 13:47:36.959787 20538 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 13:47:36.961817 20538 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 13:47:36.961941 20538 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 13:47:36.965762 20538 rpc_server.cc:307] RPC server started. Bound to: 127.13.149.253:38973
I20260924 13:47:36.965799 20605 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.149.253:38973 every 8 connection(s)
I20260924 13:47:36.966120 20538 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790257629041645-13911-0/minicluster-data/master-1/data/info.pb
I20260924 13:47:36.967099 20606 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 13:47:36.968773 13911 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJN9WRF/build/release/bin/kudu as pid 20538
I20260924 13:47:36.968703 20606 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.149.254" port: 35859 } has no permanent_uuid. Determining permanent_uuid...
Sep 24 13:47:36 dist-test-slave-sjl2 krb5kdc[20029](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257656, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.149.253@KRBTEST.COM for oryx/127.13.149.254@KRBTEST.COM
Sep 24 13:47:36 dist-test-slave-sjl2 krb5kdc[20029](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257656, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.13.149.253@KRBTEST.COM
I20260924 13:47:36.977764 20113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:36.969527 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34286 (local address 127.13.149.254:35859)
0924 13:47:36.969579 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0924 13:47:36.969581 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:47:36.969768 (+ 187us) server_negotiation.cc:407] Connection header received
0924 13:47:36.970228 (+ 460us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:47:36.970230 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:47:36.970242 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:47:36.970290 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:47:36.970847 (+ 557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:36.971334 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:36.972237 (+ 903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:36.972425 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:36.972479 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:47:36.974749 (+ 2270us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:47:36.974751 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:47:36.974752 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 13:47:36.974762 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:47:36.976122 (+ 1360us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:47:36.976583 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:47:36.976584 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:47:36.976585 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:47:36.976642 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:47:36.976901 (+ 259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:47:36.976902 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:47:36.976902 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:47:36.977149 (+ 247us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:47:36.977212 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0924 13:47:36.977576 (+ 364us) server_negotiation.cc:299] Negotiation successful
0924 13:47:36.977666 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260924 13:47:36.977963 20609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:36.969430 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37300 (local address 127.13.149.253:38973)
0924 13:47:36.969631 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0924 13:47:36.969640 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0924 13:47:36.969669 (+ 29us) server_negotiation.cc:407] Connection header received
0924 13:47:36.970105 (+ 436us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:47:36.970112 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:47:36.970138 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:47:36.970232 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:47:36.970819 (+ 587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:36.971602 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:36.972303 (+ 701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:36.972604 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:36.972661 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:47:36.975123 (+ 2462us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:47:36.975133 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:47:36.975134 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 13:47:36.975141 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:47:36.976605 (+ 1464us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:47:36.976953 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:47:36.976956 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:47:36.976957 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:47:36.977008 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:47:36.977181 (+ 173us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:47:36.977182 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:47:36.977182 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:47:36.977414 (+ 232us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:47:36.977558 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0924 13:47:36.977728 (+ 170us) server_negotiation.cc:299] Negotiation successful
0924 13:47:36.977797 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":25,"server-negotiator.queue_time_us":156,"thread_start_us":96,"threads_started":1}
I20260924 13:47:36.978160 20610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:36.969431 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:35859 (local address 127.0.0.1:34286)
0924 13:47:36.969649 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0924 13:47:36.969687 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0924 13:47:36.970129 (+ 442us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:47:36.970316 (+ 187us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:47:36.970319 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:47:36.970597 (+ 278us) client_negotiation.cc:189] Negotiated authn=SASL
0924 13:47:36.970763 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:47:36.970768 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:47:36.971420 (+ 652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:47:36.971421 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:47:36.972158 (+ 737us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:47:36.972160 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:47:36.972492 (+ 332us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:47:36.972493 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:47:36.972617 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:47:36.973119 (+ 502us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 13:47:36.973127 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0924 13:47:36.974615 (+ 1488us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 13:47:36.976211 (+ 1596us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 13:47:36.976219 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 13:47:36.976221 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 13:47:36.976466 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 13:47:36.976724 (+ 258us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 13:47:36.976725 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 13:47:36.976725 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 13:47:36.976840 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 13:47:36.977213 (+ 373us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 13:47:36.977219 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 13:47:36.977409 (+ 190us) client_negotiation.cc:769] Sending connection context
0924 13:47:36.977503 (+ 94us) client_negotiation.cc:240] Negotiation successful
0924 13:47:36.977676 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":183,"mutex_wait_us":3,"thread_start_us":139,"threads_started":1}
I20260924 13:47:36.978684 20606 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.149.253" port: 38973 } has no permanent_uuid. Determining permanent_uuid...
Sep 24 13:47:36 dist-test-slave-sjl2 krb5kdc[20029](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257656, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.149.253@KRBTEST.COM for oryx/127.13.149.253@KRBTEST.COM
I20260924 13:47:36.984555 20609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:36.978954 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37310 (local address 127.13.149.253:38973)
0924 13:47:36.978979 (+ 25us) server_negotiation.cc:206] Beginning negotiation
0924 13:47:36.978981 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:47:36.979014 (+ 33us) server_negotiation.cc:407] Connection header received
0924 13:47:36.979075 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:47:36.979076 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:47:36.979084 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:47:36.979126 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:47:36.979373 (+ 247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:36.979850 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:36.980478 (+ 628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:36.980685 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:36.980748 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:47:36.982515 (+ 1767us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:47:36.982517 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:47:36.982517 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 13:47:36.982524 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:47:36.983562 (+ 1038us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:47:36.983846 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:47:36.983847 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:47:36.983847 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:47:36.983894 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:47:36.984041 (+ 147us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:47:36.984042 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:47:36.984042 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:47:36.984163 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:47:36.984215 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0924 13:47:36.984351 (+ 136us) server_negotiation.cc:299] Negotiation successful
0924 13:47:36.984484 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":68,"server-negotiator.queue_time_us":12}
I20260924 13:47:36.984553 20610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:36.978928 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.253:38973 (local address 127.0.0.1:37310)
0924 13:47:36.978972 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0924 13:47:36.978982 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 13:47:36.979028 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:47:36.979126 (+ 98us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:47:36.979127 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:47:36.979278 (+ 151us) client_negotiation.cc:189] Negotiated authn=SASL
0924 13:47:36.979312 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:47:36.979314 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:47:36.979919 (+ 605us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:47:36.979920 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:47:36.980407 (+ 487us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:47:36.980409 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:47:36.980750 (+ 341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:47:36.980751 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:47:36.980806 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:47:36.981234 (+ 428us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 13:47:36.981238 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0924 13:47:36.982427 (+ 1189us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 13:47:36.983635 (+ 1208us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 13:47:36.983636 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 13:47:36.983637 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 13:47:36.983790 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 13:47:36.983943 (+ 153us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 13:47:36.983944 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 13:47:36.983944 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 13:47:36.983993 (+ 49us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 13:47:36.984229 (+ 236us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 13:47:36.984230 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 13:47:36.984267 (+ 37us) client_negotiation.cc:769] Sending connection context
0924 13:47:36.984325 (+ 58us) client_negotiation.cc:240] Negotiation successful
0924 13:47:36.984438 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":25,"mutex_wait_us":3}
I20260924 13:47:36.986032 20606 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1d4faf110d7a41cc8a2113c5b74a298a: Bootstrap starting.
I20260924 13:47:36.986924 20606 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1d4faf110d7a41cc8a2113c5b74a298a: Neither blocks nor log segments found. Creating new log.
I20260924 13:47:36.987205 20606 log.cc:824] T 00000000000000000000000000000000 P 1d4faf110d7a41cc8a2113c5b74a298a: Log is configured to *not* fsync() on all Append() calls
I20260924 13:47:36.988261 20606 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1d4faf110d7a41cc8a2113c5b74a298a: No bootstrap required, opened a new log
I20260924 13:47:36.989916 20606 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1d4faf110d7a41cc8a2113c5b74a298a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9976d5567e674e309eecdd83a7ced13a" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 35859 } } peers { permanent_uuid: "1d4faf110d7a41cc8a2113c5b74a298a" member_type: VOTER last_known_addr { host: "127.13.149.253" port: 38973 } }
I20260924 13:47:36.990080 20606 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1d4faf110d7a41cc8a2113c5b74a298a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 13:47:36.990103 20606 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1d4faf110d7a41cc8a2113c5b74a298a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1d4faf110d7a41cc8a2113c5b74a298a, State: Initialized, Role: FOLLOWER
I20260924 13:47:36.990260 20606 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1d4faf110d7a41cc8a2113c5b74a298a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9976d5567e674e309eecdd83a7ced13a" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 35859 } } peers { permanent_uuid: "1d4faf110d7a41cc8a2113c5b74a298a" member_type: VOTER last_known_addr { host: "127.13.149.253" port: 38973 } }
I20260924 13:47:36.990567 20612 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1d4faf110d7a41cc8a2113c5b74a298a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9976d5567e674e309eecdd83a7ced13a" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 35859 } } peers { permanent_uuid: "1d4faf110d7a41cc8a2113c5b74a298a" member_type: VOTER last_known_addr { host: "127.13.149.253" port: 38973 } } }
I20260924 13:47:36.990626 20606 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1d4faf110d7a41cc8a2113c5b74a298a [sys.catalog]: configured and running, proceeding with master startup.
I20260924 13:47:36.990671 20612 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1d4faf110d7a41cc8a2113c5b74a298a [sys.catalog]: This master's current role is: FOLLOWER
W20260924 13:47:36.992244 20623 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 1d4faf110d7a41cc8a2113c5b74a298a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260924 13:47:36.992312 20623 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260924 13:47:36.993937 20538 master_runner.cc:428] Detected that this master 1d4faf110d7a41cc8a2113c5b74a298a is joining an existing cluster
I20260924 13:47:36.993985 20538 master_runner.cc:432] Initiating AddMaster RPC to add 127.13.149.253:38973
I20260924 13:47:37.001686 20113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:36.994964 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34288 (local address 127.13.149.254:35859)
0924 13:47:36.995134 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0924 13:47:36.995136 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:47:36.995313 (+ 177us) server_negotiation.cc:407] Connection header received
0924 13:47:36.995423 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:47:36.995424 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:47:36.995436 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:47:36.995484 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:47:36.995935 (+ 451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:36.996715 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:36.997583 (+ 868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:36.997783 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:36.997834 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:47:36.998843 (+ 1009us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:47:36.998844 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:47:36.998845 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 13:47:36.998853 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:47:37.000405 (+ 1552us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:47:37.000827 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:47:37.000828 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:47:37.000828 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:47:37.000882 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:47:37.001140 (+ 258us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:47:37.001141 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:47:37.001141 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:47:37.001265 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:47:37.001329 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0924 13:47:37.001497 (+ 168us) server_negotiation.cc:299] Negotiation successful
0924 13:47:37.001591 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":140}
I20260924 13:47:37.001689 20609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:36.995176 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37316 (local address 127.13.149.253:38973)
0924 13:47:36.995214 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0924 13:47:36.995217 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 13:47:36.995315 (+ 98us) server_negotiation.cc:407] Connection header received
0924 13:47:36.995367 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:47:36.995368 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:47:36.995377 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:47:36.995424 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:47:36.995973 (+ 549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:36.996705 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:36.997869 (+ 1164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:36.998147 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:36.998183 (+ 36us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:47:36.999137 (+ 954us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:47:36.999139 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:47:36.999139 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 13:47:36.999146 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:47:37.000376 (+ 1230us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:47:37.000716 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:47:37.000718 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:47:37.000718 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:47:37.000766 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:47:37.001090 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:47:37.001091 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:47:37.001092 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:47:37.001219 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:47:37.001295 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0924 13:47:37.001432 (+ 137us) server_negotiation.cc:299] Negotiation successful
0924 13:47:37.001591 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":85,"server-negotiator.queue_time_us":21}
I20260924 13:47:37.001693 20630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:36.994931 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.253:38973 (local address 127.0.0.1:37316)
0924 13:47:36.995237 (+ 306us) negotiation.cc:107] Waiting for socket to connect
0924 13:47:36.995246 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0924 13:47:36.995318 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:47:36.995461 (+ 143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:47:36.995462 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:47:36.995748 (+ 286us) client_negotiation.cc:189] Negotiated authn=SASL
0924 13:47:36.995910 (+ 162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:47:36.995913 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:47:36.996797 (+ 884us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:47:36.996798 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:47:36.997775 (+ 977us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:47:36.997778 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:47:36.998248 (+ 470us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:47:36.998249 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:47:36.998303 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:47:36.998678 (+ 375us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 13:47:36.998683 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 13:47:36.999059 (+ 376us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 13:47:37.000456 (+ 1397us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 13:47:37.000457 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 13:47:37.000457 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 13:47:37.000661 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 13:47:37.000976 (+ 315us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 13:47:37.000976 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 13:47:37.000977 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 13:47:37.001042 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 13:47:37.001282 (+ 240us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 13:47:37.001283 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 13:47:37.001332 (+ 49us) client_negotiation.cc:769] Sending connection context
0924 13:47:37.001402 (+ 70us) client_negotiation.cc:240] Negotiation successful
0924 13:47:37.001528 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":284,"mutex_wait_us":106,"thread_start_us":96,"threads_started":1}
I20260924 13:47:37.002378 20629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:36.994922 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:35859 (local address 127.0.0.1:34288)
0924 13:47:36.995248 (+ 326us) negotiation.cc:107] Waiting for socket to connect
0924 13:47:36.995257 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0924 13:47:36.995339 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:47:36.995487 (+ 148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:47:36.995487 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:47:36.995742 (+ 255us) client_negotiation.cc:189] Negotiated authn=SASL
0924 13:47:36.995868 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:47:36.995871 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:47:36.996795 (+ 924us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:47:36.996796 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:47:36.997492 (+ 696us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:47:36.997494 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:47:36.997850 (+ 356us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:47:36.997851 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:47:36.997912 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:47:36.998340 (+ 428us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 13:47:36.998345 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 13:47:36.998760 (+ 415us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 13:47:37.000500 (+ 1740us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 13:47:37.000501 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 13:47:37.000501 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 13:47:37.000760 (+ 259us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 13:47:37.000977 (+ 217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 13:47:37.000978 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 13:47:37.000979 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 13:47:37.001075 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 13:47:37.001353 (+ 278us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 13:47:37.001354 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 13:47:37.001387 (+ 33us) client_negotiation.cc:769] Sending connection context
0924 13:47:37.001771 (+ 384us) client_negotiation.cc:240] Negotiation successful
0924 13:47:37.001880 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"mutex_wait_us":44,"thread_start_us":91,"threads_started":1}
Sep 24 13:47:37 dist-test-slave-sjl2 krb5kdc[20029](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257656, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.149.254@KRBTEST.COM for oryx/127.13.149.253@KRBTEST.COM
I20260924 13:47:37.012758 20631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:37.004812 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.253:38973 (local address 127.0.0.1:37320)
0924 13:47:37.004983 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0924 13:47:37.005031 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0924 13:47:37.005112 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:47:37.005262 (+ 150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:47:37.005265 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:47:37.005644 (+ 379us) client_negotiation.cc:189] Negotiated authn=SASL
0924 13:47:37.005715 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:47:37.005718 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:47:37.006364 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:47:37.006365 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:47:37.007354 (+ 989us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:47:37.007357 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:47:37.007692 (+ 335us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:47:37.007693 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:47:37.007784 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:47:37.008311 (+ 527us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 13:47:37.008316 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 13:47:37.009912 (+ 1596us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 13:47:37.011605 (+ 1693us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 13:47:37.011617 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 13:47:37.011619 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 13:47:37.011846 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 13:47:37.012043 (+ 197us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 13:47:37.012043 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 13:47:37.012043 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 13:47:37.012095 (+ 52us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 13:47:37.012382 (+ 287us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 13:47:37.012391 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 13:47:37.012449 (+ 58us) client_negotiation.cc:769] Sending connection context
0924 13:47:37.012550 (+ 101us) client_negotiation.cc:240] Negotiation successful
0924 13:47:37.012664 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":86,"threads_started":1}
I20260924 13:47:37.012768 20609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:37.004904 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37320 (local address 127.13.149.253:38973)
0924 13:47:37.005042 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0924 13:47:37.005044 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:47:37.005083 (+ 39us) server_negotiation.cc:407] Connection header received
0924 13:47:37.005192 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:47:37.005193 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:47:37.005202 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:47:37.005256 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:47:37.005819 (+ 563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:37.006286 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:37.007441 (+ 1155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:37.007623 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:37.007678 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:47:37.010010 (+ 2332us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:47:37.010012 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:47:37.010013 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 13:47:37.010021 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:47:37.011513 (+ 1492us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:47:37.011922 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:47:37.011924 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:47:37.011924 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:47:37.011986 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:47:37.012141 (+ 155us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:47:37.012142 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:47:37.012142 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:47:37.012281 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:47:37.012344 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0924 13:47:37.012575 (+ 231us) server_negotiation.cc:299] Negotiation successful
0924 13:47:37.012671 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":107}
I20260924 13:47:37.013391 20074 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.13.149.253:38973: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "1d4faf110d7a41cc8a2113c5b74a298a"
member_type: NON_VOTER
last_known_addr {
host: "127.13.149.253"
port: 38973
}
attrs {
promote: true
}
}
dest_uuid: "9976d5567e674e309eecdd83a7ced13a"
cas_config_opid_index: -1
I20260924 13:47:37.013768 20074 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9976d5567e674e309eecdd83a7ced13a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "9976d5567e674e309eecdd83a7ced13a" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 35859 } } peers { permanent_uuid: "1d4faf110d7a41cc8a2113c5b74a298a" member_type: NON_VOTER last_known_addr { host: "127.13.149.253" port: 38973 } attrs { promote: true } }
I20260924 13:47:37.014328 20633 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 9976d5567e674e309eecdd83a7ced13a [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 1d4faf110d7a41cc8a2113c5b74a298a (127.13.149.253) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "9976d5567e674e309eecdd83a7ced13a" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 35859 } } peers { permanent_uuid: "1d4faf110d7a41cc8a2113c5b74a298a" member_type: NON_VOTER last_known_addr { host: "127.13.149.253" port: 38973 } attrs { promote: true } } }
I20260924 13:47:37.014637 20580 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1d4faf110d7a41cc8a2113c5b74a298a [term 0 FOLLOWER]: Advancing to term 1
I20260924 13:47:37.015416 20633 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.13.149.253:38973
I20260924 13:47:37.015506 20580 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 1d4faf110d7a41cc8a2113c5b74a298a [term 1 FOLLOWER]: Refusing update from remote peer 9976d5567e674e309eecdd83a7ced13a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260924 13:47:37.015506 20633 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9976d5567e674e309eecdd83a7ced13a [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "9976d5567e674e309eecdd83a7ced13a" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "9976d5567e674e309eecdd83a7ced13a" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 35859 } } peers { permanent_uuid: "1d4faf110d7a41cc8a2113c5b74a298a" member_type: NON_VOTER last_known_addr { host: "127.13.149.253" port: 38973 } attrs { promote: true } } }
I20260924 13:47:37.015563 20633 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9976d5567e674e309eecdd83a7ced13a [sys.catalog]: This master's current role is: LEADER
I20260924 13:47:37.015827 20538 master.cc:537] Master@127.13.149.253:38973 shutting down...
I20260924 13:47:37.015978 20632 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 9976d5567e674e309eecdd83a7ced13a [LEADER]: Connected to new peer: Peer: permanent_uuid: "1d4faf110d7a41cc8a2113c5b74a298a" member_type: NON_VOTER last_known_addr { host: "127.13.149.253" port: 38973 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
W20260924 13:47:37.016783 20059 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
W20260924 13:47:37.017264 20059 consensus_peers.cc:487] T 00000000000000000000000000000000 P 9976d5567e674e309eecdd83a7ced13a -> Peer 1d4faf110d7a41cc8a2113c5b74a298a (127.13.149.253:38973): Couldn't send request to peer 1d4faf110d7a41cc8a2113c5b74a298a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260924 13:47:37.018725 20538 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 1d4faf110d7a41cc8a2113c5b74a298a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 13:47:37.018811 20538 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 1d4faf110d7a41cc8a2113c5b74a298a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 13:47:37.018847 20538 tablet_replica.cc:330] T 00000000000000000000000000000000 P 1d4faf110d7a41cc8a2113c5b74a298a: stopping tablet replica
I20260924 13:47:37.031828 20538 master.cc:552] Master@127.13.149.253:38973 shutdown complete.
I20260924 13:47:37.031895 20538 master_runner.cc:305] Clearing existing system tablet
I20260924 13:47:37.032024 20538 fs_manager.cc:359] Metadata directory not provided
I20260924 13:47:37.032070 20538 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 13:47:37.032891 20538 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260924 13:47:37.033480 20637 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 13:47:37.033598 20538 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790257629041645-13911-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 13:47:37.033651 20538 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260924 13:47:37.033669 20538 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790257629041645-13911-0/minicluster-data/master-1/data,/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790257629041645-13911-0/minicluster-data/master-1/wal
uuid: "1d4faf110d7a41cc8a2113c5b74a298a"
format_stamp: "Formatted at 2026-09-24 13:47:36 on dist-test-slave-sjl2"
server_key: "a60161f45e778623c5e6b6f536b04d81"
server_key_iv: "338522ce603bcd6a49c50315f1afa730"
server_key_version: "encryptionkey@0"
I20260924 13:47:37.034343 20538 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 1d4faf110d7a41cc8a2113c5b74a298a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 13:47:37.035847 20538 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 1d4faf110d7a41cc8a2113c5b74a298a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260924 13:47:37.035923 20538 log.cc:1196] T 00000000000000000000000000000000 P 1d4faf110d7a41cc8a2113c5b74a298a: Deleting WAL directory at /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790257629041645-13911-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260924 13:47:37.036123 20538 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 1d4faf110d7a41cc8a2113c5b74a298a: Deleting consensus metadata
I20260924 13:47:37.036229 20538 master_runner.cc:315] Copying system tablet from 127.13.149.254:35859
I20260924 13:47:37.036769 20538 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 1d4faf110d7a41cc8a2113c5b74a298a: tablet copy: Beginning tablet copy session from remote peer at address 127.13.149.254:35859
I20260924 13:47:37.042465 20113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:37.037180 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34294 (local address 127.13.149.254:35859)
0924 13:47:37.037232 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0924 13:47:37.037234 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:47:37.037325 (+ 91us) server_negotiation.cc:407] Connection header received
0924 13:47:37.037398 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:47:37.037399 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:47:37.037410 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:47:37.037457 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:47:37.037767 (+ 310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:37.038259 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:37.038972 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:37.039166 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:37.039223 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:47:37.040143 (+ 920us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:47:37.040144 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:47:37.040145 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 13:47:37.040153 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:47:37.041436 (+ 1283us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:47:37.041748 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:47:37.041750 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:47:37.041750 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:47:37.041803 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:47:37.041969 (+ 166us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:47:37.041970 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:47:37.041970 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:47:37.042084 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:47:37.042145 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0924 13:47:37.042279 (+ 134us) server_negotiation.cc:299] Negotiation successful
0924 13:47:37.042363 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260924 13:47:37.042482 20643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:37.037122 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:35859 (local address 127.0.0.1:34294)
0924 13:47:37.037251 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0924 13:47:37.037261 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 13:47:37.037336 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:47:37.037461 (+ 125us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:47:37.037461 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:47:37.037636 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0924 13:47:37.037692 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:47:37.037694 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:47:37.038332 (+ 638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:47:37.038333 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:47:37.038893 (+ 560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:47:37.038895 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:47:37.039229 (+ 334us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:47:37.039231 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:47:37.039301 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:47:37.039742 (+ 441us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 13:47:37.039747 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 13:47:37.040045 (+ 298us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 13:47:37.041518 (+ 1473us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 13:47:37.041520 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 13:47:37.041520 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 13:47:37.041688 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 13:47:37.041855 (+ 167us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 13:47:37.041855 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 13:47:37.041856 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 13:47:37.041911 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 13:47:37.042142 (+ 231us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 13:47:37.042143 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 13:47:37.042192 (+ 49us) client_negotiation.cc:769] Sending connection context
0924 13:47:37.042251 (+ 59us) client_negotiation.cc:240] Negotiation successful
0924 13:47:37.042364 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":69,"threads_started":1}
I20260924 13:47:37.042999 20094 tablet_copy_service.cc:140] P 9976d5567e674e309eecdd83a7ced13a: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 1d4faf110d7a41cc8a2113c5b74a298a ({username='oryx', principal='oryx/127.13.149.253@KRBTEST.COM'} at 127.0.0.1:34294)
I20260924 13:47:37.043094 20094 tablet_copy_service.cc:161] P 9976d5567e674e309eecdd83a7ced13a: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 1d4faf110d7a41cc8a2113c5b74a298a at {username='oryx', principal='oryx/127.13.149.253@KRBTEST.COM'} at 127.0.0.1:34294: session id = 1d4faf110d7a41cc8a2113c5b74a298a-00000000000000000000000000000000
I20260924 13:47:37.043747 20094 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 9976d5567e674e309eecdd83a7ced13a: Tablet Copy: opened 0 blocks and 1 log segments
I20260924 13:47:37.044387 20538 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260924 13:47:37.045485 20538 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 1d4faf110d7a41cc8a2113c5b74a298a: tablet copy: Starting download of 0 data blocks...
I20260924 13:47:37.045610 20538 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 1d4faf110d7a41cc8a2113c5b74a298a: tablet copy: Starting download of 1 WAL segments...
I20260924 13:47:37.046447 20538 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 1d4faf110d7a41cc8a2113c5b74a298a: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260924 13:47:37.047495 20094 tablet_copy_service.cc:342] P 9976d5567e674e309eecdd83a7ced13a: Request end of tablet copy session 1d4faf110d7a41cc8a2113c5b74a298a-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.13.149.253@KRBTEST.COM'} at 127.0.0.1:34294
I20260924 13:47:37.047564 20094 tablet_copy_service.cc:433] P 9976d5567e674e309eecdd83a7ced13a: ending tablet copy session 1d4faf110d7a41cc8a2113c5b74a298a-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 1d4faf110d7a41cc8a2113c5b74a298a
I20260924 13:47:37.050333 20538 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 13:47:37.051627 20644 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 13:47:37.051676 20647 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 13:47:37.051688 20645 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 13:47:37.051877 20538 server_base.cc:1034] running on GCE node
I20260924 13:47:37.052076 20538 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 13:47:37.052222 20538 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 13:47:37.053354 20538 hybrid_clock.cc:648] HybridClock initialized: now 1790257657053346 us; error 26 us; skew 500 ppm
Sep 24 13:47:37 dist-test-slave-sjl2 krb5kdc[20029](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257657, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.149.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 13:47:37.054601 20538 init.cc:377] Logged in from keytab as oryx/127.13.149.253@KRBTEST.COM (short username oryx)
I20260924 13:47:37.054848 20538 webserver.cc:460] Webserver started at http://127.13.149.253:35517/ using document root <none> and password file <none>
I20260924 13:47:37.054963 20538 fs_manager.cc:359] Metadata directory not provided
I20260924 13:47:37.054992 20538 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 13:47:37.055572 20538 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.000s sys 0.000s
I20260924 13:47:37.056037 20654 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 13:47:37.056157 20538 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260924 13:47:37.056196 20538 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790257629041645-13911-0/minicluster-data/master-1/data,/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790257629041645-13911-0/minicluster-data/master-1/wal
uuid: "1d4faf110d7a41cc8a2113c5b74a298a"
format_stamp: "Formatted at 2026-09-24 13:47:36 on dist-test-slave-sjl2"
server_key: "a60161f45e778623c5e6b6f536b04d81"
server_key_iv: "338522ce603bcd6a49c50315f1afa730"
server_key_version: "encryptionkey@0"
I20260924 13:47:37.056255 20538 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790257629041645-13911-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790257629041645-13911-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790257629041645-13911-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 13:47:37.071972 20538 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 13:47:37.073439 20538 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 13:47:37.076062 20538 rpc_server.cc:307] RPC server started. Bound to: 127.13.149.253:38973
I20260924 13:47:37.076077 20706 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.149.253:38973 every 8 connection(s)
I20260924 13:47:37.076370 20538 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790257629041645-13911-0/minicluster-data/master-1/data/info.pb
I20260924 13:47:37.077250 20707 sys_catalog.cc:263] Verifying existing consensus state
I20260924 13:47:37.077811 20707 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1d4faf110d7a41cc8a2113c5b74a298a: Bootstrap starting.
I20260924 13:47:37.081122 20707 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1d4faf110d7a41cc8a2113c5b74a298a: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260924 13:47:37.081354 20707 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1d4faf110d7a41cc8a2113c5b74a298a: Bootstrap complete.
I20260924 13:47:37.081554 20707 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1d4faf110d7a41cc8a2113c5b74a298a [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "9976d5567e674e309eecdd83a7ced13a" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 35859 } } peers { permanent_uuid: "1d4faf110d7a41cc8a2113c5b74a298a" member_type: NON_VOTER last_known_addr { host: "127.13.149.253" port: 38973 } attrs { promote: true } }
I20260924 13:47:37.081658 20707 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1d4faf110d7a41cc8a2113c5b74a298a [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 1d4faf110d7a41cc8a2113c5b74a298a, State: Initialized, Role: LEARNER
I20260924 13:47:37.081697 20707 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1d4faf110d7a41cc8a2113c5b74a298a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "9976d5567e674e309eecdd83a7ced13a" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 35859 } } peers { permanent_uuid: "1d4faf110d7a41cc8a2113c5b74a298a" member_type: NON_VOTER last_known_addr { host: "127.13.149.253" port: 38973 } attrs { promote: true } }
I20260924 13:47:37.081892 20707 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1d4faf110d7a41cc8a2113c5b74a298a [sys.catalog]: configured and running, proceeding with master startup.
I20260924 13:47:37.081876 20711 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1d4faf110d7a41cc8a2113c5b74a298a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "9976d5567e674e309eecdd83a7ced13a" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 35859 } } peers { permanent_uuid: "1d4faf110d7a41cc8a2113c5b74a298a" member_type: NON_VOTER last_known_addr { host: "127.13.149.253" port: 38973 } attrs { promote: true } } }
I20260924 13:47:37.082065 20711 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1d4faf110d7a41cc8a2113c5b74a298a [sys.catalog]: This master's current role is: LEARNER
I20260924 13:47:37.083911 20722 catalog_manager.cc:1259] Loaded cluster ID: 1a89448487764376bbc82fc8eb6bb72c
I20260924 13:47:37.083966 20722 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 1d4faf110d7a41cc8a2113c5b74a298a: loading cluster ID for follower catalog manager: success
I20260924 13:47:37.085114 20722 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 1d4faf110d7a41cc8a2113c5b74a298a: acquiring CA information for follower catalog manager: success
I20260924 13:47:37.085438 20722 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 1d4faf110d7a41cc8a2113c5b74a298a: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260924 13:47:37.087002 20710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:37.079711 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37338 (local address 127.13.149.253:38973)
0924 13:47:37.079841 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0924 13:47:37.079843 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:47:37.079915 (+ 72us) server_negotiation.cc:407] Connection header received
0924 13:47:37.079990 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:47:37.079991 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:47:37.079999 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:47:37.080064 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:47:37.080574 (+ 510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:37.081339 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:37.082263 (+ 924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:37.082560 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:37.082627 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:47:37.083935 (+ 1308us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:47:37.083940 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:47:37.083941 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 13:47:37.083948 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:47:37.085876 (+ 1928us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:47:37.086290 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:47:37.086290 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:47:37.086291 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:47:37.086345 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:47:37.086536 (+ 191us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:47:37.086537 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:47:37.086537 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:47:37.086644 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:47:37.086717 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0924 13:47:37.086851 (+ 134us) server_negotiation.cc:299] Negotiation successful
0924 13:47:37.086925 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":105,"thread_start_us":83,"threads_started":1}
WARNING: no policy specified for oryx/127.13.149.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.13.149.252@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.13.149.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790257629041645-13911-0/krb5kdc/oryx_127.13.149.252.keytab.
Entry for principal oryx/127.13.149.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790257629041645-13911-0/krb5kdc/oryx_127.13.149.252.keytab.
I20260924 13:47:37.112464 13911 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJN9WRF/build/release/bin/kudu
/tmp/dist-test-taskJN9WRF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790257629041645-13911-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790257629041645-13911-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790257629041645-13911-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790257629041645-13911-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.149.252:35717
--webserver_interface=127.13.149.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790257629041645-13911-0/krb5kdc/oryx_127.13.149.252.keytab
--principal=oryx/127.13.149.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.149.212:44199
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.149.254:35859,127.13.149.253:38973,127.13.149.252:35717
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790257629041645-13911-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790257629041645-13911-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790257629041645-13911-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 13:47:37.195709 20725 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 13:47:37.195943 20725 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 13:47:37.195968 20725 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 13:47:37.197222 20725 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 13:47:37.197269 20725 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 13:47:37.197285 20725 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 13:47:37.197299 20725 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 13:47:37.197311 20725 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 13:47:37.198894 20725 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.149.212:44199
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790257629041645-13911-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790257629041645-13911-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.13.149.254:35859,127.13.149.253:38973,127.13.149.252:35717
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790257629041645-13911-0/krb5kdc/oryx_127.13.149.252.keytab
--principal=oryx/127.13.149.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.149.252:35717
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790257629041645-13911-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.13.149.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20725
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790257629041645-13911-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 12:29:37 UTC on ccycloud.quasar-iqzbxn.root.comops.site
build id 4670106
I20260924 13:47:37.199133 20725 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 13:47:37.199332 20725 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260924 13:47:37.204317 20725 server_base.cc:1034] running on GCE node
W20260924 13:47:37.204288 20731 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 13:47:37.204281 20730 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 13:47:37.204298 20733 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 13:47:37.204793 20725 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 13:47:37.205143 20725 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 13:47:37.206311 20725 hybrid_clock.cc:648] HybridClock initialized: now 1790257657206294 us; error 33 us; skew 500 ppm
Sep 24 13:47:37 dist-test-slave-sjl2 krb5kdc[20029](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257657, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.149.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 13:47:37.208782 20725 init.cc:377] Logged in from keytab as oryx/127.13.149.252@KRBTEST.COM (short username oryx)
I20260924 13:47:37.209272 20725 webserver.cc:460] Webserver started at http://127.13.149.252:43957/ using document root <none> and password file <none>
I20260924 13:47:37.209457 20725 fs_manager.cc:359] Metadata directory not provided
I20260924 13:47:37.209496 20725 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 13:47:37.209591 20725 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 13:47:37.210603 20725 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790257629041645-13911-0/minicluster-data/master-2/data/instance:
uuid: "e42b5e67f6b6495080300913733a80db"
format_stamp: "Formatted at 2026-09-24 13:47:37 on dist-test-slave-sjl2"
server_key: "4e4895f795062438aa96d3b039717891"
server_key_iv: "986b3d75abdcd9e284d8632004d6f2ae"
server_key_version: "encryptionkey@0"
I20260924 13:47:37.210940 20725 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790257629041645-13911-0/minicluster-data/master-2/wal/instance:
uuid: "e42b5e67f6b6495080300913733a80db"
format_stamp: "Formatted at 2026-09-24 13:47:37 on dist-test-slave-sjl2"
server_key: "4e4895f795062438aa96d3b039717891"
server_key_iv: "986b3d75abdcd9e284d8632004d6f2ae"
server_key_version: "encryptionkey@0"
I20260924 13:47:37.212170 20725 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260924 13:47:37.212874 20740 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 13:47:37.213069 20725 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260924 13:47:37.213177 20725 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790257629041645-13911-0/minicluster-data/master-2/data,/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790257629041645-13911-0/minicluster-data/master-2/wal
uuid: "e42b5e67f6b6495080300913733a80db"
format_stamp: "Formatted at 2026-09-24 13:47:37 on dist-test-slave-sjl2"
server_key: "4e4895f795062438aa96d3b039717891"
server_key_iv: "986b3d75abdcd9e284d8632004d6f2ae"
server_key_version: "encryptionkey@0"
I20260924 13:47:37.213259 20725 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790257629041645-13911-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790257629041645-13911-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790257629041645-13911-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 13:47:37.234182 20725 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 13:47:37.236400 20725 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 13:47:37.236527 20725 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 13:47:37.240366 20725 rpc_server.cc:307] RPC server started. Bound to: 127.13.149.252:35717
I20260924 13:47:37.240401 20792 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.149.252:35717 every 8 connection(s)
I20260924 13:47:37.240741 20725 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790257629041645-13911-0/minicluster-data/master-2/data/info.pb
I20260924 13:47:37.241703 20793 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 13:47:37.243283 20793 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.149.254" port: 35859 } has no permanent_uuid. Determining permanent_uuid...
I20260924 13:47:37.247851 13911 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJN9WRF/build/release/bin/kudu as pid 20725
Sep 24 13:47:37 dist-test-slave-sjl2 krb5kdc[20029](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257657, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.149.252@KRBTEST.COM for oryx/127.13.149.254@KRBTEST.COM
I20260924 13:47:37.253075 20113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:37.244202 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34306 (local address 127.13.149.254:35859)
0924 13:47:37.244318 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0924 13:47:37.244321 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 13:47:37.244349 (+ 28us) server_negotiation.cc:407] Connection header received
0924 13:47:37.244840 (+ 491us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:47:37.244841 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:47:37.244851 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:47:37.244904 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:47:37.245579 (+ 675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:37.246141 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:37.247147 (+ 1006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:37.247349 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:37.247406 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:47:37.250010 (+ 2604us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:47:37.250012 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:47:37.250012 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 13:47:37.250022 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:47:37.251604 (+ 1582us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:47:37.252114 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:47:37.252115 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:47:37.252116 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:47:37.252180 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:47:37.252408 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:47:37.252408 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:47:37.252409 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:47:37.252522 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:47:37.252579 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0924 13:47:37.252868 (+ 289us) server_negotiation.cc:299] Negotiation successful
0924 13:47:37.252977 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260924 13:47:37.253293 20795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:37.244012 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:35859 (local address 127.0.0.1:34306)
0924 13:47:37.244232 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0924 13:47:37.244281 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0924 13:47:37.244757 (+ 476us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:47:37.245006 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:47:37.245009 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:47:37.245277 (+ 268us) client_negotiation.cc:189] Negotiated authn=SASL
0924 13:47:37.245494 (+ 217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:47:37.245499 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:47:37.246254 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:47:37.246255 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:47:37.247065 (+ 810us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:47:37.247067 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:47:37.247424 (+ 357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:47:37.247425 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:47:37.247527 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:47:37.248092 (+ 565us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 13:47:37.248101 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0924 13:47:37.249870 (+ 1769us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 13:47:37.251715 (+ 1845us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 13:47:37.251722 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 13:47:37.251724 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 13:47:37.252043 (+ 319us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 13:47:37.252237 (+ 194us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 13:47:37.252237 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 13:47:37.252237 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 13:47:37.252353 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 13:47:37.252576 (+ 223us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 13:47:37.252582 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 13:47:37.252729 (+ 147us) client_negotiation.cc:769] Sending connection context
0924 13:47:37.252837 (+ 108us) client_negotiation.cc:240] Negotiation successful
0924 13:47:37.253040 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":113,"threads_started":1}
Sep 24 13:47:37 dist-test-slave-sjl2 krb5kdc[20029](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257656, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.13.149.252@KRBTEST.COM
I20260924 13:47:37.253875 20793 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.149.253" port: 38973 } has no permanent_uuid. Determining permanent_uuid...
I20260924 13:47:37.256999 20796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:37.248548 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53874 (local address 127.13.149.252:35717)
0924 13:47:37.248745 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0924 13:47:37.248749 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 13:47:37.248769 (+ 20us) server_negotiation.cc:407] Connection header received
0924 13:47:37.248787 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:47:37.248788 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:47:37.248805 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:47:37.248877 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:47:37.249577 (+ 700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:37.250408 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:37.251164 (+ 756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:37.251486 (+ 322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:37.251554 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:47:37.253910 (+ 2356us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:47:37.253915 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:47:37.253915 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 13:47:37.253921 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:47:37.255550 (+ 1629us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:47:37.256016 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:47:37.256020 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:47:37.256020 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:47:37.256138 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:47:37.256341 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:47:37.256342 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:47:37.256342 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:47:37.256530 (+ 188us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:47:37.256690 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0924 13:47:37.256763 (+ 73us) server_negotiation.cc:299] Negotiation successful
0924 13:47:37.256853 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":164,"thread_start_us":98,"threads_started":1}
Sep 24 13:47:37 dist-test-slave-sjl2 krb5kdc[20029](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257657, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.149.252@KRBTEST.COM for oryx/127.13.149.253@KRBTEST.COM
I20260924 13:47:37.261468 20795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:37.254130 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.253:38973 (local address 127.0.0.1:37348)
0924 13:47:37.254173 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0924 13:47:37.254184 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0924 13:47:37.254219 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:47:37.254349 (+ 130us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:47:37.254350 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:47:37.254494 (+ 144us) client_negotiation.cc:189] Negotiated authn=SASL
0924 13:47:37.254549 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:47:37.254550 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:47:37.255193 (+ 643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:47:37.255194 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:47:37.255849 (+ 655us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:47:37.255853 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:47:37.256387 (+ 534us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:47:37.256388 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:47:37.256456 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:47:37.257014 (+ 558us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 13:47:37.257018 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0924 13:47:37.258408 (+ 1390us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 13:47:37.260276 (+ 1868us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 13:47:37.260277 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 13:47:37.260277 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 13:47:37.260460 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 13:47:37.260728 (+ 268us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 13:47:37.260730 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 13:47:37.260730 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 13:47:37.260794 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 13:47:37.261178 (+ 384us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 13:47:37.261179 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 13:47:37.261227 (+ 48us) client_negotiation.cc:769] Sending connection context
0924 13:47:37.261300 (+ 73us) client_negotiation.cc:240] Negotiation successful
0924 13:47:37.261395 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":26}
I20260924 13:47:37.261476 20710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:37.254207 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37348 (local address 127.13.149.253:38973)
0924 13:47:37.254258 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0924 13:47:37.254260 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:47:37.254268 (+ 8us) server_negotiation.cc:407] Connection header received
0924 13:47:37.254291 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:47:37.254292 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:47:37.254300 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:47:37.254343 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:47:37.254616 (+ 273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:37.255120 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:37.255999 (+ 879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:37.256313 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:37.256383 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:47:37.258508 (+ 2125us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:47:37.258510 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:47:37.258510 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 13:47:37.258521 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:47:37.260197 (+ 1676us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:47:37.260570 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:47:37.260571 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:47:37.260571 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:47:37.260635 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:47:37.260901 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:47:37.260902 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:47:37.260903 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:47:37.261064 (+ 161us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:47:37.261136 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0924 13:47:37.261324 (+ 188us) server_negotiation.cc:299] Negotiation successful
0924 13:47:37.261409 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260924 13:47:37.261863 20793 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.149.252" port: 35717 } has no permanent_uuid. Determining permanent_uuid...
Sep 24 13:47:37 dist-test-slave-sjl2 krb5kdc[20029](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257657, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.149.252@KRBTEST.COM for oryx/127.13.149.252@KRBTEST.COM
I20260924 13:47:37.268260 20795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:37.262116 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.252:35717 (local address 127.0.0.1:53880)
0924 13:47:37.262161 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0924 13:47:37.262171 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 13:47:37.262206 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:47:37.262327 (+ 121us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:47:37.262327 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:47:37.262439 (+ 112us) client_negotiation.cc:189] Negotiated authn=SASL
0924 13:47:37.262479 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:47:37.262480 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:47:37.263070 (+ 590us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:47:37.263070 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:47:37.263571 (+ 501us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:47:37.263572 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:47:37.263968 (+ 396us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:47:37.263970 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:47:37.264017 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:47:37.264458 (+ 441us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 13:47:37.264462 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0924 13:47:37.265741 (+ 1279us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 13:47:37.267249 (+ 1508us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 13:47:37.267250 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 13:47:37.267250 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 13:47:37.267419 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 13:47:37.267650 (+ 231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 13:47:37.267650 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 13:47:37.267651 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 13:47:37.267713 (+ 62us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 13:47:37.267966 (+ 253us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 13:47:37.267967 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 13:47:37.268009 (+ 42us) client_negotiation.cc:769] Sending connection context
0924 13:47:37.268063 (+ 54us) client_negotiation.cc:240] Negotiation successful
0924 13:47:37.268161 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30}
I20260924 13:47:37.268307 20796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:37.262159 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53880 (local address 127.13.149.252:35717)
0924 13:47:37.262206 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0924 13:47:37.262207 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0924 13:47:37.262214 (+ 7us) server_negotiation.cc:407] Connection header received
0924 13:47:37.262261 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:47:37.262262 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:47:37.262272 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:47:37.262316 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:47:37.262525 (+ 209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:37.263012 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:37.263670 (+ 658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:37.263873 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:37.263941 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:47:37.265846 (+ 1905us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:47:37.265848 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:47:37.265848 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 13:47:37.265855 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:47:37.267148 (+ 1293us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:47:37.267493 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:47:37.267494 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:47:37.267495 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:47:37.267560 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:47:37.267771 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:47:37.267772 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:47:37.267772 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:47:37.267916 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:47:37.267965 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0924 13:47:37.268101 (+ 136us) server_negotiation.cc:299] Negotiation successful
0924 13:47:37.268208 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":39,"server-negotiator.queue_time_us":30}
I20260924 13:47:37.269774 20793 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e42b5e67f6b6495080300913733a80db: Bootstrap starting.
I20260924 13:47:37.270567 20793 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e42b5e67f6b6495080300913733a80db: Neither blocks nor log segments found. Creating new log.
I20260924 13:47:37.270820 20793 log.cc:824] T 00000000000000000000000000000000 P e42b5e67f6b6495080300913733a80db: Log is configured to *not* fsync() on all Append() calls
I20260924 13:47:37.271488 20793 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e42b5e67f6b6495080300913733a80db: No bootstrap required, opened a new log
I20260924 13:47:37.273030 20793 raft_consensus.cc:348] T 00000000000000000000000000000000 P e42b5e67f6b6495080300913733a80db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9976d5567e674e309eecdd83a7ced13a" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 35859 } } peers { permanent_uuid: "1d4faf110d7a41cc8a2113c5b74a298a" member_type: VOTER last_known_addr { host: "127.13.149.253" port: 38973 } } peers { permanent_uuid: "e42b5e67f6b6495080300913733a80db" member_type: VOTER last_known_addr { host: "127.13.149.252" port: 35717 } }
I20260924 13:47:37.273171 20793 raft_consensus.cc:374] T 00000000000000000000000000000000 P e42b5e67f6b6495080300913733a80db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 13:47:37.273203 20793 raft_consensus.cc:729] T 00000000000000000000000000000000 P e42b5e67f6b6495080300913733a80db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e42b5e67f6b6495080300913733a80db, State: Initialized, Role: FOLLOWER
I20260924 13:47:37.273331 20793 consensus_queue.cc:260] T 00000000000000000000000000000000 P e42b5e67f6b6495080300913733a80db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9976d5567e674e309eecdd83a7ced13a" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 35859 } } peers { permanent_uuid: "1d4faf110d7a41cc8a2113c5b74a298a" member_type: VOTER last_known_addr { host: "127.13.149.253" port: 38973 } } peers { permanent_uuid: "e42b5e67f6b6495080300913733a80db" member_type: VOTER last_known_addr { host: "127.13.149.252" port: 35717 } }
I20260924 13:47:37.273613 20793 sys_catalog.cc:564] T 00000000000000000000000000000000 P e42b5e67f6b6495080300913733a80db [sys.catalog]: configured and running, proceeding with master startup.
I20260924 13:47:37.273587 20798 sys_catalog.cc:455] T 00000000000000000000000000000000 P e42b5e67f6b6495080300913733a80db [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9976d5567e674e309eecdd83a7ced13a" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 35859 } } peers { permanent_uuid: "1d4faf110d7a41cc8a2113c5b74a298a" member_type: VOTER last_known_addr { host: "127.13.149.253" port: 38973 } } peers { permanent_uuid: "e42b5e67f6b6495080300913733a80db" member_type: VOTER last_known_addr { host: "127.13.149.252" port: 35717 } } }
I20260924 13:47:37.273667 20798 sys_catalog.cc:458] T 00000000000000000000000000000000 P e42b5e67f6b6495080300913733a80db [sys.catalog]: This master's current role is: FOLLOWER
W20260924 13:47:37.275136 20809 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e42b5e67f6b6495080300913733a80db: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260924 13:47:37.275198 20809 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260924 13:47:37.277347 20725 master_runner.cc:428] Detected that this master e42b5e67f6b6495080300913733a80db is joining an existing cluster
I20260924 13:47:37.277405 20725 master_runner.cc:432] Initiating AddMaster RPC to add 127.13.149.252:35717
I20260924 13:47:37.283227 20798 raft_consensus.cc:482] T 00000000000000000000000000000000 P e42b5e67f6b6495080300913733a80db [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 13:47:37.283342 20798 raft_consensus.cc:504] T 00000000000000000000000000000000 P e42b5e67f6b6495080300913733a80db [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9976d5567e674e309eecdd83a7ced13a" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 35859 } } peers { permanent_uuid: "1d4faf110d7a41cc8a2113c5b74a298a" member_type: VOTER last_known_addr { host: "127.13.149.253" port: 38973 } } peers { permanent_uuid: "e42b5e67f6b6495080300913733a80db" member_type: VOTER last_known_addr { host: "127.13.149.252" port: 35717 } }
I20260924 13:47:37.283609 20798 leader_election.cc:288] T 00000000000000000000000000000000 P e42b5e67f6b6495080300913733a80db [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9976d5567e674e309eecdd83a7ced13a (127.13.149.254:35859), 1d4faf110d7a41cc8a2113c5b74a298a (127.13.149.253:38973)
I20260924 13:47:37.283870 20681 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e42b5e67f6b6495080300913733a80db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1d4faf110d7a41cc8a2113c5b74a298a" is_pre_election: true
I20260924 13:47:37.283988 20681 raft_consensus.cc:1799] T 00000000000000000000000000000000 P 1d4faf110d7a41cc8a2113c5b74a298a [term 1 LEARNER]: Handling vote request from an unknown peer e42b5e67f6b6495080300913733a80db
I20260924 13:47:37.284044 20681 raft_consensus.cc:2396] T 00000000000000000000000000000000 P 1d4faf110d7a41cc8a2113c5b74a298a [term 1 LEARNER]: Leader pre-election vote request: Denying vote to candidate e42b5e67f6b6495080300913733a80db for term 1 because replica has last-logged OpId of term: 1 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 0 index: 0.
I20260924 13:47:37.284165 20084 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e42b5e67f6b6495080300913733a80db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9976d5567e674e309eecdd83a7ced13a" is_pre_election: true
I20260924 13:47:37.284834 20742 leader_election.cc:302] T 00000000000000000000000000000000 P e42b5e67f6b6495080300913733a80db [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e42b5e67f6b6495080300913733a80db; no voters: 1d4faf110d7a41cc8a2113c5b74a298a, 9976d5567e674e309eecdd83a7ced13a
I20260924 13:47:37.285005 20798 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e42b5e67f6b6495080300913733a80db [term 0 FOLLOWER]: Advancing to term 1
I20260924 13:47:37.285957 20798 raft_consensus.cc:2726] T 00000000000000000000000000000000 P e42b5e67f6b6495080300913733a80db [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260924 13:47:37.287335 20815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:37.278586 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.253:38973 (local address 127.0.0.1:37354)
0924 13:47:37.278738 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0924 13:47:37.278750 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0924 13:47:37.278816 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:47:37.278990 (+ 174us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:47:37.278991 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:47:37.279209 (+ 218us) client_negotiation.cc:189] Negotiated authn=SASL
0924 13:47:37.279289 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:47:37.279291 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:47:37.280265 (+ 974us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:47:37.280266 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:47:37.281559 (+ 1293us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:47:37.281562 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:47:37.282103 (+ 541us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:47:37.282103 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:47:37.282166 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:47:37.282706 (+ 540us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 13:47:37.282712 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0924 13:47:37.283244 (+ 532us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 13:47:37.286393 (+ 3149us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 13:47:37.286394 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 13:47:37.286395 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 13:47:37.286599 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 13:47:37.286794 (+ 195us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 13:47:37.286794 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 13:47:37.286795 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 13:47:37.286851 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 13:47:37.287092 (+ 241us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 13:47:37.287093 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 13:47:37.287129 (+ 36us) client_negotiation.cc:769] Sending connection context
0924 13:47:37.287172 (+ 43us) client_negotiation.cc:240] Negotiation successful
0924 13:47:37.287260 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"mutex_wait_us":23,"thread_start_us":86,"threads_started":1}
I20260924 13:47:37.287379 20710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:37.278494 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37354 (local address 127.13.149.253:38973)
0924 13:47:37.278539 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0924 13:47:37.278541 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:47:37.278819 (+ 278us) server_negotiation.cc:407] Connection header received
0924 13:47:37.278870 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:47:37.278871 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:47:37.278882 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:47:37.278941 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:47:37.279394 (+ 453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:37.280155 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:37.281658 (+ 1503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:37.281960 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:37.282005 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:47:37.283387 (+ 1382us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:47:37.283388 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:47:37.283389 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 13:47:37.283397 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:47:37.286294 (+ 2897us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:47:37.286666 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:47:37.286667 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:47:37.286668 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:47:37.286734 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:47:37.286900 (+ 166us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:47:37.286901 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:47:37.286901 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:47:37.287036 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:47:37.287096 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0924 13:47:37.287206 (+ 110us) server_negotiation.cc:299] Negotiation successful
0924 13:47:37.287286 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260924 13:47:37.308722 13911 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJN9WRF/build/release/bin/kudu with pid 20136
I20260924 13:47:37.314359 13911 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJN9WRF/build/release/bin/kudu
/tmp/dist-test-taskJN9WRF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790257629041645-13911-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790257629041645-13911-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790257629041645-13911-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790257629041645-13911-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.149.193:44033
--local_ip_for_outbound_sockets=127.13.149.193
--tserver_master_addrs=127.13.149.254:35859
--webserver_port=34119
--webserver_interface=127.13.149.193
--keytab_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790257629041645-13911-0/krb5kdc/oryx_127.13.149.193.keytab
--principal=oryx/127.13.149.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.149.212:44199
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.13.149.254:35859,127.13.149.253:38973
--tserver_master_addrs=127.13.149.254:35859,127.13.149.253:38973,127.13.149.252:35717 with env {KRB5CCNAME=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790257629041645-13911-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790257629041645-13911-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790257629041645-13911-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260924 13:47:37.315037 20113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:37.278494 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34322 (local address 127.13.149.254:35859)
0924 13:47:37.278539 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0924 13:47:37.278541 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:47:37.279103 (+ 562us) server_negotiation.cc:407] Connection header received
0924 13:47:37.279138 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:47:37.279140 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:47:37.279151 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:47:37.279203 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:47:37.279669 (+ 466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:37.280410 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:37.281642 (+ 1232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:37.281937 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:37.281978 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:47:37.311192 (+ 29214us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:47:37.311194 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:47:37.311195 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 13:47:37.311206 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:47:37.312841 (+ 1635us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:47:37.313960 (+ 1119us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:47:37.313962 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:47:37.313962 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:47:37.314032 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:47:37.314256 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:47:37.314257 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:47:37.314258 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:47:37.314426 (+ 168us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:47:37.314500 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0924 13:47:37.314691 (+ 191us) server_negotiation.cc:299] Negotiation successful
0924 13:47:37.314786 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260924 13:47:37.315362 20817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:37.278735 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:35859 (local address 127.0.0.1:34322)
0924 13:47:37.279031 (+ 296us) negotiation.cc:107] Waiting for socket to connect
0924 13:47:37.279039 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0924 13:47:37.279094 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:47:37.279249 (+ 155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:47:37.279250 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:47:37.279507 (+ 257us) client_negotiation.cc:189] Negotiated authn=SASL
0924 13:47:37.279581 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:47:37.279583 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:47:37.280500 (+ 917us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:47:37.280501 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:47:37.281551 (+ 1050us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:47:37.281553 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:47:37.282175 (+ 622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:47:37.282176 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:47:37.282271 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:47:37.310548 (+ 28277us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 13:47:37.310556 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0924 13:47:37.311039 (+ 483us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 13:47:37.312983 (+ 1944us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 13:47:37.312984 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 13:47:37.312984 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 13:47:37.313873 (+ 889us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 13:47:37.314102 (+ 229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 13:47:37.314103 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 13:47:37.314103 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 13:47:37.314160 (+ 57us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 13:47:37.314533 (+ 373us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 13:47:37.314534 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 13:47:37.314584 (+ 50us) client_negotiation.cc:769] Sending connection context
0924 13:47:37.315130 (+ 546us) client_negotiation.cc:240] Negotiation successful
0924 13:47:37.315261 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":285,"mutex_wait_us":1,"thread_start_us":72,"threads_started":1}
I20260924 13:47:37.317029 20796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:37.278954 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53894 (local address 127.13.149.252:35717)
0924 13:47:37.278988 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0924 13:47:37.278990 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:47:37.279201 (+ 211us) server_negotiation.cc:407] Connection header received
0924 13:47:37.279270 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:47:37.279270 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:47:37.279278 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:47:37.279301 (+ 23us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:47:37.279664 (+ 363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:37.280409 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:37.284681 (+ 4272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:37.284959 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:37.285013 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:47:37.311963 (+ 26950us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:47:37.311965 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:47:37.311965 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 13:47:37.311972 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:47:37.313703 (+ 1731us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:47:37.315997 (+ 2294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:47:37.315998 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:47:37.315998 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:47:37.316059 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:47:37.316251 (+ 192us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:47:37.316252 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:47:37.316252 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:47:37.316358 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:47:37.316399 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0924 13:47:37.316810 (+ 411us) server_negotiation.cc:299] Negotiation successful
0924 13:47:37.316946 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260924 13:47:37.317340 20816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:37.278925 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.252:35717 (local address 127.0.0.1:53894)
0924 13:47:37.279160 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0924 13:47:37.279167 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0924 13:47:37.279216 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:47:37.279363 (+ 147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:47:37.279363 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:47:37.279553 (+ 190us) client_negotiation.cc:189] Negotiated authn=SASL
0924 13:47:37.279597 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:47:37.279598 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:47:37.283847 (+ 4249us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:47:37.283848 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:47:37.284615 (+ 767us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:47:37.284617 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:47:37.285061 (+ 444us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:47:37.285061 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:47:37.285118 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:47:37.311207 (+ 26089us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 13:47:37.311211 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0924 13:47:37.311863 (+ 652us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 13:47:37.313797 (+ 1934us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 13:47:37.313799 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 13:47:37.313799 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 13:47:37.315925 (+ 2126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 13:47:37.316139 (+ 214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 13:47:37.316139 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 13:47:37.316140 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 13:47:37.316195 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 13:47:37.316429 (+ 234us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 13:47:37.316430 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 13:47:37.316464 (+ 34us) client_negotiation.cc:769] Sending connection context
0924 13:47:37.317146 (+ 682us) client_negotiation.cc:240] Negotiation successful
0924 13:47:37.317245 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":224,"thread_start_us":54,"threads_started":1}
Sep 24 13:47:37 dist-test-slave-sjl2 krb5kdc[20029](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257656, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.149.254@KRBTEST.COM for oryx/127.13.149.252@KRBTEST.COM
I20260924 13:47:37.325996 20796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:37.317936 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53898 (local address 127.13.149.252:35717)
0924 13:47:37.317970 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0924 13:47:37.317972 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:47:37.317980 (+ 8us) server_negotiation.cc:407] Connection header received
0924 13:47:37.318012 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:47:37.318013 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:47:37.318023 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:47:37.318078 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:47:37.318379 (+ 301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:37.319107 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:37.320111 (+ 1004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:37.320420 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:37.320493 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:47:37.322811 (+ 2318us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:47:37.322813 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:47:37.322813 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 13:47:37.322822 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:47:37.324724 (+ 1902us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:47:37.325167 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:47:37.325168 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:47:37.325169 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:47:37.325233 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:47:37.325428 (+ 195us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:47:37.325429 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:47:37.325429 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:47:37.325578 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:47:37.325650 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0924 13:47:37.325811 (+ 161us) server_negotiation.cc:299] Negotiation successful
0924 13:47:37.325894 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260924 13:47:37.326007 20631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:37.317838 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.252:35717 (local address 127.0.0.1:53898)
0924 13:47:37.317884 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0924 13:47:37.317894 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 13:47:37.317934 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:47:37.318090 (+ 156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:47:37.318091 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:47:37.318256 (+ 165us) client_negotiation.cc:189] Negotiated authn=SASL
0924 13:47:37.318311 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:47:37.318312 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:47:37.319198 (+ 886us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:47:37.319200 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:47:37.320020 (+ 820us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:47:37.320023 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:47:37.320501 (+ 478us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:47:37.320502 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:47:37.320577 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:47:37.321095 (+ 518us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 13:47:37.321101 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0924 13:47:37.322706 (+ 1605us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 13:47:37.324826 (+ 2120us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 13:47:37.324827 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 13:47:37.324828 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 13:47:37.325075 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 13:47:37.325302 (+ 227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 13:47:37.325302 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 13:47:37.325303 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 13:47:37.325369 (+ 66us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 13:47:37.325645 (+ 276us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 13:47:37.325646 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 13:47:37.325702 (+ 56us) client_negotiation.cc:769] Sending connection context
0924 13:47:37.325774 (+ 72us) client_negotiation.cc:240] Negotiation successful
0924 13:47:37.325900 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":23}
I20260924 13:47:37.326985 20074 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.13.149.252:35717: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "e42b5e67f6b6495080300913733a80db"
member_type: NON_VOTER
last_known_addr {
host: "127.13.149.252"
port: 35717
}
attrs {
promote: true
}
}
dest_uuid: "9976d5567e674e309eecdd83a7ced13a"
cas_config_opid_index: 5
I20260924 13:47:37.327248 20074 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9976d5567e674e309eecdd83a7ced13a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "9976d5567e674e309eecdd83a7ced13a" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 35859 } } peers { permanent_uuid: "1d4faf110d7a41cc8a2113c5b74a298a" member_type: NON_VOTER last_known_addr { host: "127.13.149.253" port: 38973 } attrs { promote: true } } peers { permanent_uuid: "e42b5e67f6b6495080300913733a80db" member_type: NON_VOTER last_known_addr { host: "127.13.149.252" port: 35717 } attrs { promote: true } }
I20260924 13:47:37.327622 20632 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 9976d5567e674e309eecdd83a7ced13a [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER e42b5e67f6b6495080300913733a80db (127.13.149.252) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "9976d5567e674e309eecdd83a7ced13a" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 35859 } } peers { permanent_uuid: "1d4faf110d7a41cc8a2113c5b74a298a" member_type: NON_VOTER last_known_addr { host: "127.13.149.253" port: 38973 } attrs { promote: true } } peers { permanent_uuid: "e42b5e67f6b6495080300913733a80db" member_type: NON_VOTER last_known_addr { host: "127.13.149.252" port: 35717 } attrs { promote: true } } }
I20260924 13:47:37.328600 20632 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.13.149.252:35717
I20260924 13:47:37.328691 20632 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9976d5567e674e309eecdd83a7ced13a [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "9976d5567e674e309eecdd83a7ced13a" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "9976d5567e674e309eecdd83a7ced13a" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 35859 } } peers { permanent_uuid: "1d4faf110d7a41cc8a2113c5b74a298a" member_type: NON_VOTER last_known_addr { host: "127.13.149.253" port: 38973 } attrs { promote: true } } peers { permanent_uuid: "e42b5e67f6b6495080300913733a80db" member_type: NON_VOTER last_known_addr { host: "127.13.149.252" port: 35717 } attrs { promote: true } } }
I20260924 13:47:37.328756 20632 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9976d5567e674e309eecdd83a7ced13a [sys.catalog]: This master's current role is: LEADER
I20260924 13:47:37.329375 20767 raft_consensus.cc:1261] T 00000000000000000000000000000000 P e42b5e67f6b6495080300913733a80db [term 1 FOLLOWER]: Refusing update from remote peer 9976d5567e674e309eecdd83a7ced13a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260924 13:47:37.329488 20725 master.cc:537] Master@127.13.149.252:35717 shutting down...
I20260924 13:47:37.330039 20633 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 9976d5567e674e309eecdd83a7ced13a [LEADER]: Connected to new peer: Peer: permanent_uuid: "e42b5e67f6b6495080300913733a80db" member_type: NON_VOTER last_known_addr { host: "127.13.149.252" port: 35717 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.000s
W20260924 13:47:37.330765 20061 consensus_peers.cc:487] T 00000000000000000000000000000000 P 9976d5567e674e309eecdd83a7ced13a -> Peer e42b5e67f6b6495080300913733a80db (127.13.149.252:35717): Couldn't send request to peer e42b5e67f6b6495080300913733a80db. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260924 13:47:37.332664 20725 raft_consensus.cc:2229] T 00000000000000000000000000000000 P e42b5e67f6b6495080300913733a80db [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 13:47:37.332757 20725 raft_consensus.cc:2258] T 00000000000000000000000000000000 P e42b5e67f6b6495080300913733a80db [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 13:47:37.332798 20725 tablet_replica.cc:330] T 00000000000000000000000000000000 P e42b5e67f6b6495080300913733a80db: stopping tablet replica
I20260924 13:47:37.334074 20710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:37.327796 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37368 (local address 127.13.149.253:38973)
0924 13:47:37.327834 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0924 13:47:37.327835 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0924 13:47:37.327910 (+ 75us) server_negotiation.cc:407] Connection header received
0924 13:47:37.328001 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:47:37.328001 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:47:37.328025 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:47:37.328067 (+ 42us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 13:47:37.328298 (+ 231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:37.330551 (+ 2253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:37.332904 (+ 2353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:37.333560 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:37.333615 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:47:37.333685 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0924 13:47:37.333940 (+ 255us) server_negotiation.cc:299] Negotiation successful
0924 13:47:37.333988 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260924 13:47:37.334383 20631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:37.327698 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.253:38973 (local address 127.0.0.1:37368)
0924 13:47:37.327867 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0924 13:47:37.327876 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0924 13:47:37.327933 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:47:37.328144 (+ 211us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:47:37.328145 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:47:37.328146 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 13:47:37.328223 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:47:37.328225 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:47:37.330659 (+ 2434us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:47:37.330660 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:47:37.332732 (+ 2072us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:47:37.332738 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:47:37.333751 (+ 1013us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:47:37.333752 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:47:37.333822 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:47:37.333859 (+ 37us) client_negotiation.cc:769] Sending connection context
0924 13:47:37.334147 (+ 288us) client_negotiation.cc:240] Negotiation successful
0924 13:47:37.334312 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":156}
I20260924 13:47:37.334630 20681 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 1d4faf110d7a41cc8a2113c5b74a298a [term 1 LEARNER]: Refusing update from remote peer 9976d5567e674e309eecdd83a7ced13a: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260924 13:47:37.334827 20633 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 9976d5567e674e309eecdd83a7ced13a [LEADER]: Connected to new peer: Peer: permanent_uuid: "1d4faf110d7a41cc8a2113c5b74a298a" member_type: NON_VOTER last_known_addr { host: "127.13.149.253" port: 38973 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260924 13:47:37.335523 20681 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 1d4faf110d7a41cc8a2113c5b74a298a [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER e42b5e67f6b6495080300913733a80db (127.13.149.252) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "9976d5567e674e309eecdd83a7ced13a" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 35859 } } peers { permanent_uuid: "1d4faf110d7a41cc8a2113c5b74a298a" member_type: NON_VOTER last_known_addr { host: "127.13.149.253" port: 38973 } attrs { promote: true } } peers { permanent_uuid: "e42b5e67f6b6495080300913733a80db" member_type: NON_VOTER last_known_addr { host: "127.13.149.252" port: 35717 } attrs { promote: true } } }
I20260924 13:47:37.336669 20711 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1d4faf110d7a41cc8a2113c5b74a298a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9976d5567e674e309eecdd83a7ced13a. Latest consensus state: current_term: 1 leader_uuid: "9976d5567e674e309eecdd83a7ced13a" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "9976d5567e674e309eecdd83a7ced13a" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 35859 } } peers { permanent_uuid: "1d4faf110d7a41cc8a2113c5b74a298a" member_type: NON_VOTER last_known_addr { host: "127.13.149.253" port: 38973 } attrs { promote: true } } peers { permanent_uuid: "e42b5e67f6b6495080300913733a80db" member_type: NON_VOTER last_known_addr { host: "127.13.149.252" port: 35717 } attrs { promote: true } } }
I20260924 13:47:37.336755 20711 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1d4faf110d7a41cc8a2113c5b74a298a [sys.catalog]: This master's current role is: LEARNER
I20260924 13:47:37.336784 20711 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1d4faf110d7a41cc8a2113c5b74a298a [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "9976d5567e674e309eecdd83a7ced13a" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "9976d5567e674e309eecdd83a7ced13a" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 35859 } } peers { permanent_uuid: "1d4faf110d7a41cc8a2113c5b74a298a" member_type: NON_VOTER last_known_addr { host: "127.13.149.253" port: 38973 } attrs { promote: true } } peers { permanent_uuid: "e42b5e67f6b6495080300913733a80db" member_type: NON_VOTER last_known_addr { host: "127.13.149.252" port: 35717 } attrs { promote: true } } }
I20260924 13:47:37.336831 20711 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1d4faf110d7a41cc8a2113c5b74a298a [sys.catalog]: This master's current role is: LEARNER
I20260924 13:47:37.337203 20633 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 9976d5567e674e309eecdd83a7ced13a: attempting to promote NON_VOTER 1d4faf110d7a41cc8a2113c5b74a298a to VOTER
I20260924 13:47:37.337414 20633 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9976d5567e674e309eecdd83a7ced13a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "9976d5567e674e309eecdd83a7ced13a" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 35859 } } peers { permanent_uuid: "1d4faf110d7a41cc8a2113c5b74a298a" member_type: VOTER last_known_addr { host: "127.13.149.253" port: 38973 } attrs { promote: false } } peers { permanent_uuid: "e42b5e67f6b6495080300913733a80db" member_type: NON_VOTER last_known_addr { host: "127.13.149.252" port: 35717 } attrs { promote: true } }
I20260924 13:47:37.337801 20725 master.cc:552] Master@127.13.149.252:35717 shutdown complete.
I20260924 13:47:37.337837 20725 master_runner.cc:305] Clearing existing system tablet
I20260924 13:47:37.337961 20725 fs_manager.cc:359] Metadata directory not provided
I20260924 13:47:37.337994 20725 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 13:47:37.338105 20681 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 1d4faf110d7a41cc8a2113c5b74a298a [term 1 LEARNER]: Refusing update from remote peer 9976d5567e674e309eecdd83a7ced13a: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260924 13:47:37.338260 20632 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 9976d5567e674e309eecdd83a7ced13a [LEADER]: Connected to new peer: Peer: permanent_uuid: "1d4faf110d7a41cc8a2113c5b74a298a" member_type: VOTER last_known_addr { host: "127.13.149.253" port: 38973 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260924 13:47:37.338656 20631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:37.337792 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.252:35717 (local address 127.0.0.1:53910)
0924 13:47:37.338575 (+ 783us) negotiation.cc:107] Waiting for socket to connect
0924 13:47:37.338622 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.149.252:35717: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":769,"thread_start_us":51,"threads_started":1}
I20260924 13:47:37.338981 20631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:37.338898 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.252:35717 (local address 127.0.0.1:53924)
0924 13:47:37.338932 (+ 34us) negotiation.cc:107] Waiting for socket to connect
0924 13:47:37.338952 (+ 20us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.149.252:35717: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
W20260924 13:47:37.339027 20061 consensus_peers.cc:487] T 00000000000000000000000000000000 P 9976d5567e674e309eecdd83a7ced13a -> Peer e42b5e67f6b6495080300913733a80db (127.13.149.252:35717): Couldn't send request to peer e42b5e67f6b6495080300913733a80db. Status: Network error: Client connection negotiation failed: client connection to 127.13.149.252:35717: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260924 13:47:37.339133 20632 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 9976d5567e674e309eecdd83a7ced13a [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 1d4faf110d7a41cc8a2113c5b74a298a (127.13.149.253) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "9976d5567e674e309eecdd83a7ced13a" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 35859 } } peers { permanent_uuid: "1d4faf110d7a41cc8a2113c5b74a298a" member_type: VOTER last_known_addr { host: "127.13.149.253" port: 38973 } attrs { promote: false } } peers { permanent_uuid: "e42b5e67f6b6495080300913733a80db" member_type: NON_VOTER last_known_addr { host: "127.13.149.252" port: 35717 } attrs { promote: true } } }
I20260924 13:47:37.340090 20632 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9976d5567e674e309eecdd83a7ced13a [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "9976d5567e674e309eecdd83a7ced13a" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "9976d5567e674e309eecdd83a7ced13a" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 35859 } } peers { permanent_uuid: "1d4faf110d7a41cc8a2113c5b74a298a" member_type: VOTER last_known_addr { host: "127.13.149.253" port: 38973 } attrs { promote: false } } peers { permanent_uuid: "e42b5e67f6b6495080300913733a80db" member_type: NON_VOTER last_known_addr { host: "127.13.149.252" port: 35717 } attrs { promote: true } } }
I20260924 13:47:37.340166 20632 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9976d5567e674e309eecdd83a7ced13a [sys.catalog]: This master's current role is: LEADER
I20260924 13:47:37.340302 20633 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9976d5567e674e309eecdd83a7ced13a [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "9976d5567e674e309eecdd83a7ced13a" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "9976d5567e674e309eecdd83a7ced13a" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 35859 } } peers { permanent_uuid: "1d4faf110d7a41cc8a2113c5b74a298a" member_type: VOTER last_known_addr { host: "127.13.149.253" port: 38973 } attrs { promote: false } } peers { permanent_uuid: "e42b5e67f6b6495080300913733a80db" member_type: NON_VOTER last_known_addr { host: "127.13.149.252" port: 35717 } attrs { promote: true } } }
I20260924 13:47:37.340386 20633 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9976d5567e674e309eecdd83a7ced13a [sys.catalog]: This master's current role is: LEADER
I20260924 13:47:37.340444 20681 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 1d4faf110d7a41cc8a2113c5b74a298a [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 1d4faf110d7a41cc8a2113c5b74a298a (127.13.149.253) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "9976d5567e674e309eecdd83a7ced13a" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 35859 } } peers { permanent_uuid: "1d4faf110d7a41cc8a2113c5b74a298a" member_type: VOTER last_known_addr { host: "127.13.149.253" port: 38973 } attrs { promote: false } } peers { permanent_uuid: "e42b5e67f6b6495080300913733a80db" member_type: NON_VOTER last_known_addr { host: "127.13.149.252" port: 35717 } attrs { promote: true } } }
I20260924 13:47:37.341217 20725 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.000s sys 0.001s
I20260924 13:47:37.342504 20820 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1d4faf110d7a41cc8a2113c5b74a298a [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "9976d5567e674e309eecdd83a7ced13a" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "9976d5567e674e309eecdd83a7ced13a" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 35859 } } peers { permanent_uuid: "1d4faf110d7a41cc8a2113c5b74a298a" member_type: VOTER last_known_addr { host: "127.13.149.253" port: 38973 } attrs { promote: false } } peers { permanent_uuid: "e42b5e67f6b6495080300913733a80db" member_type: NON_VOTER last_known_addr { host: "127.13.149.252" port: 35717 } attrs { promote: true } } }
I20260924 13:47:37.342590 20820 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1d4faf110d7a41cc8a2113c5b74a298a [sys.catalog]: This master's current role is: FOLLOWER
I20260924 13:47:37.342700 20825 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 13:47:37.342811 20725 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790257629041645-13911-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 13:47:37.342875 20725 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260924 13:47:37.342917 20725 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790257629041645-13911-0/minicluster-data/master-2/data,/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790257629041645-13911-0/minicluster-data/master-2/wal
uuid: "e42b5e67f6b6495080300913733a80db"
format_stamp: "Formatted at 2026-09-24 13:47:37 on dist-test-slave-sjl2"
server_key: "4e4895f795062438aa96d3b039717891"
server_key_iv: "986b3d75abdcd9e284d8632004d6f2ae"
server_key_version: "encryptionkey@0"
I20260924 13:47:37.343689 20725 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P e42b5e67f6b6495080300913733a80db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 13:47:37.345117 20725 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P e42b5e67f6b6495080300913733a80db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260924 13:47:37.345194 20725 log.cc:1196] T 00000000000000000000000000000000 P e42b5e67f6b6495080300913733a80db: Deleting WAL directory at /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790257629041645-13911-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260924 13:47:37.345373 20725 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P e42b5e67f6b6495080300913733a80db: Deleting consensus metadata
I20260924 13:47:37.345602 20725 master_runner.cc:315] Copying system tablet from 127.13.149.254:35859
I20260924 13:47:37.346199 20725 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P e42b5e67f6b6495080300913733a80db: tablet copy: Beginning tablet copy session from remote peer at address 127.13.149.254:35859
I20260924 13:47:37.353451 20113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:37.346628 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34336 (local address 127.13.149.254:35859)
0924 13:47:37.346672 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0924 13:47:37.346674 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:47:37.346950 (+ 276us) server_negotiation.cc:407] Connection header received
0924 13:47:37.347008 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:47:37.347009 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:47:37.347021 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:47:37.347093 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:47:37.347458 (+ 365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:37.348211 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:37.349245 (+ 1034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:37.349537 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:37.349604 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:47:37.350668 (+ 1064us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:47:37.350670 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:47:37.350670 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 13:47:37.350678 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:47:37.352227 (+ 1549us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:47:37.352607 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:47:37.352608 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:47:37.352609 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:47:37.352677 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:47:37.352877 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:47:37.352877 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:47:37.352878 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:47:37.353043 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:47:37.353110 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0924 13:47:37.353274 (+ 164us) server_negotiation.cc:299] Negotiation successful
0924 13:47:37.353354 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260924 13:47:37.353461 20831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:37.346719 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:35859 (local address 127.0.0.1:34336)
0924 13:47:37.346857 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0924 13:47:37.346871 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0924 13:47:37.346950 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:47:37.347113 (+ 163us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:47:37.347114 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:47:37.347319 (+ 205us) client_negotiation.cc:189] Negotiated authn=SASL
0924 13:47:37.347378 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:47:37.347381 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:47:37.348297 (+ 916us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:47:37.348298 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:47:37.349159 (+ 861us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:47:37.349162 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:47:37.349609 (+ 447us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:47:37.349610 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:47:37.349677 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:47:37.350174 (+ 497us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 13:47:37.350179 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 13:47:37.350579 (+ 400us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 13:47:37.352318 (+ 1739us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 13:47:37.352319 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 13:47:37.352320 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 13:47:37.352532 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 13:47:37.352746 (+ 214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 13:47:37.352747 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 13:47:37.352747 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 13:47:37.352814 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 13:47:37.353110 (+ 296us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 13:47:37.353111 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 13:47:37.353165 (+ 54us) client_negotiation.cc:769] Sending connection context
0924 13:47:37.353239 (+ 74us) client_negotiation.cc:240] Negotiation successful
0924 13:47:37.353354 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":75,"threads_started":1}
I20260924 13:47:37.353818 20094 tablet_copy_service.cc:140] P 9976d5567e674e309eecdd83a7ced13a: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer e42b5e67f6b6495080300913733a80db ({username='oryx', principal='oryx/127.13.149.252@KRBTEST.COM'} at 127.0.0.1:34336)
I20260924 13:47:37.353871 20094 tablet_copy_service.cc:161] P 9976d5567e674e309eecdd83a7ced13a: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer e42b5e67f6b6495080300913733a80db at {username='oryx', principal='oryx/127.13.149.252@KRBTEST.COM'} at 127.0.0.1:34336: session id = e42b5e67f6b6495080300913733a80db-00000000000000000000000000000000
I20260924 13:47:37.354461 20094 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 9976d5567e674e309eecdd83a7ced13a: Tablet Copy: opened 0 blocks and 1 log segments
I20260924 13:47:37.354931 20725 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260924 13:47:37.356426 20725 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P e42b5e67f6b6495080300913733a80db: tablet copy: Starting download of 0 data blocks...
I20260924 13:47:37.356576 20725 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P e42b5e67f6b6495080300913733a80db: tablet copy: Starting download of 1 WAL segments...
I20260924 13:47:37.357359 20725 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P e42b5e67f6b6495080300913733a80db: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260924 13:47:37.358341 20094 tablet_copy_service.cc:342] P 9976d5567e674e309eecdd83a7ced13a: Request end of tablet copy session e42b5e67f6b6495080300913733a80db-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.13.149.252@KRBTEST.COM'} at 127.0.0.1:34336
I20260924 13:47:37.358394 20094 tablet_copy_service.cc:433] P 9976d5567e674e309eecdd83a7ced13a: ending tablet copy session e42b5e67f6b6495080300913733a80db-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer e42b5e67f6b6495080300913733a80db
I20260924 13:47:37.361035 20725 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 13:47:37.362474 20835 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 13:47:37.362488 20832 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 13:47:37.362895 20833 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 13:47:37.365151 20725 server_base.cc:1034] running on GCE node
I20260924 13:47:37.365278 20725 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 13:47:37.365430 20725 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 13:47:37.366556 20725 hybrid_clock.cc:648] HybridClock initialized: now 1790257657366537 us; error 41 us; skew 500 ppm
Sep 24 13:47:37 dist-test-slave-sjl2 krb5kdc[20029](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257657, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.149.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 13:47:37.368084 20725 init.cc:377] Logged in from keytab as oryx/127.13.149.252@KRBTEST.COM (short username oryx)
I20260924 13:47:37.368369 20725 webserver.cc:460] Webserver started at http://127.13.149.252:37169/ using document root <none> and password file <none>
I20260924 13:47:37.368489 20725 fs_manager.cc:359] Metadata directory not provided
I20260924 13:47:37.368526 20725 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 13:47:37.369314 20725 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260924 13:47:37.369894 20842 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 13:47:37.369998 20725 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260924 13:47:37.370031 20725 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790257629041645-13911-0/minicluster-data/master-2/data,/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790257629041645-13911-0/minicluster-data/master-2/wal
uuid: "e42b5e67f6b6495080300913733a80db"
format_stamp: "Formatted at 2026-09-24 13:47:37 on dist-test-slave-sjl2"
server_key: "4e4895f795062438aa96d3b039717891"
server_key_iv: "986b3d75abdcd9e284d8632004d6f2ae"
server_key_version: "encryptionkey@0"
I20260924 13:47:37.370090 20725 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790257629041645-13911-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790257629041645-13911-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790257629041645-13911-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 13:47:37.385222 20725 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 13:47:37.387005 20725 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 13:47:37.390105 20725 rpc_server.cc:307] RPC server started. Bound to: 127.13.149.252:35717
I20260924 13:47:37.390512 20725 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790257629041645-13911-0/minicluster-data/master-2/data/info.pb
I20260924 13:47:37.393059 20894 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.149.252:35717 every 8 connection(s)
I20260924 13:47:37.393870 20895 sys_catalog.cc:263] Verifying existing consensus state
I20260924 13:47:37.394374 20895 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e42b5e67f6b6495080300913733a80db: Bootstrap starting.
I20260924 13:47:37.397310 20895 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e42b5e67f6b6495080300913733a80db: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260924 13:47:37.397516 20895 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e42b5e67f6b6495080300913733a80db: Bootstrap complete.
I20260924 13:47:37.397707 20895 raft_consensus.cc:348] T 00000000000000000000000000000000 P e42b5e67f6b6495080300913733a80db [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "9976d5567e674e309eecdd83a7ced13a" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 35859 } } peers { permanent_uuid: "1d4faf110d7a41cc8a2113c5b74a298a" member_type: VOTER last_known_addr { host: "127.13.149.253" port: 38973 } attrs { promote: false } } peers { permanent_uuid: "e42b5e67f6b6495080300913733a80db" member_type: NON_VOTER last_known_addr { host: "127.13.149.252" port: 35717 } attrs { promote: true } }
I20260924 13:47:37.397796 20895 raft_consensus.cc:729] T 00000000000000000000000000000000 P e42b5e67f6b6495080300913733a80db [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: e42b5e67f6b6495080300913733a80db, State: Initialized, Role: LEARNER
I20260924 13:47:37.397830 20895 consensus_queue.cc:260] T 00000000000000000000000000000000 P e42b5e67f6b6495080300913733a80db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "9976d5567e674e309eecdd83a7ced13a" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 35859 } } peers { permanent_uuid: "1d4faf110d7a41cc8a2113c5b74a298a" member_type: VOTER last_known_addr { host: "127.13.149.253" port: 38973 } attrs { promote: false } } peers { permanent_uuid: "e42b5e67f6b6495080300913733a80db" member_type: NON_VOTER last_known_addr { host: "127.13.149.252" port: 35717 } attrs { promote: true } }
I20260924 13:47:37.398018 20895 sys_catalog.cc:564] T 00000000000000000000000000000000 P e42b5e67f6b6495080300913733a80db [sys.catalog]: configured and running, proceeding with master startup.
I20260924 13:47:37.398025 20898 sys_catalog.cc:455] T 00000000000000000000000000000000 P e42b5e67f6b6495080300913733a80db [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "9976d5567e674e309eecdd83a7ced13a" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 35859 } } peers { permanent_uuid: "1d4faf110d7a41cc8a2113c5b74a298a" member_type: VOTER last_known_addr { host: "127.13.149.253" port: 38973 } attrs { promote: false } } peers { permanent_uuid: "e42b5e67f6b6495080300913733a80db" member_type: NON_VOTER last_known_addr { host: "127.13.149.252" port: 35717 } attrs { promote: true } } }
I20260924 13:47:37.398270 20898 sys_catalog.cc:458] T 00000000000000000000000000000000 P e42b5e67f6b6495080300913733a80db [sys.catalog]: This master's current role is: LEARNER
I20260924 13:47:37.399492 20909 catalog_manager.cc:1259] Loaded cluster ID: 1a89448487764376bbc82fc8eb6bb72c
I20260924 13:47:37.399591 20909 catalog_manager.cc:1552] T 00000000000000000000000000000000 P e42b5e67f6b6495080300913733a80db: loading cluster ID for follower catalog manager: success
I20260924 13:47:37.401437 20909 catalog_manager.cc:1574] T 00000000000000000000000000000000 P e42b5e67f6b6495080300913733a80db: acquiring CA information for follower catalog manager: success
I20260924 13:47:37.401746 20909 catalog_manager.cc:1602] T 00000000000000000000000000000000 P e42b5e67f6b6495080300913733a80db: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260924 13:47:37.420982 20818 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 13:47:37.421275 20818 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 13:47:37.421314 20818 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 13:47:37.422610 20818 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 13:47:37.422663 20818 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 13:47:37.422703 20818 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.149.193
I20260924 13:47:37.424324 20818 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.149.212:44199
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790257629041645-13911-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790257629041645-13911-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790257629041645-13911-0/krb5kdc/oryx_127.13.149.193.keytab
--principal=oryx/127.13.149.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.149.193:44033
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790257629041645-13911-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.149.193
--webserver_port=34119
--tserver_master_addrs=127.13.149.254:35859,127.13.149.253:38973,127.13.149.252:35717
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20818
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.149.193
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790257629041645-13911-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 12:29:37 UTC on ccycloud.quasar-iqzbxn.root.comops.site
build id 4670106
I20260924 13:47:37.424626 20818 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 13:47:37.424849 20818 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 13:47:37.430063 20914 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 13:47:37.430063 20917 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 13:47:37.430065 20915 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 13:47:37.430204 20818 server_base.cc:1034] running on GCE node
I20260924 13:47:37.430527 20818 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 13:47:37.430764 20818 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 13:47:37.431914 20818 hybrid_clock.cc:648] HybridClock initialized: now 1790257657431901 us; error 27 us; skew 500 ppm
Sep 24 13:47:37 dist-test-slave-sjl2 krb5kdc[20029](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257657, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.149.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 13:47:37.434479 20818 init.cc:377] Logged in from keytab as oryx/127.13.149.193@KRBTEST.COM (short username oryx)
I20260924 13:47:37.434893 20818 webserver.cc:460] Webserver started at http://127.13.149.193:34119/ using document root <none> and password file <none>
I20260924 13:47:37.435099 20818 fs_manager.cc:359] Metadata directory not provided
I20260924 13:47:37.435164 20818 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 13:47:37.436468 20818 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260924 13:47:37.437004 20924 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 13:47:37.437157 20818 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260924 13:47:37.437233 20818 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790257629041645-13911-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790257629041645-13911-0/minicluster-data/ts-0/wal
uuid: "b79e3cc74b9d4c54bd7e3bc4626d08a1"
format_stamp: "Formatted at 2026-09-24 13:47:36 on dist-test-slave-sjl2"
server_key: "ab8f708b7e507f334286734901d5523a"
server_key_iv: "d2e9173d948a1003087b2b6093d702f3"
server_key_version: "encryptionkey@0"
I20260924 13:47:37.437566 20818 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790257629041645-13911-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790257629041645-13911-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790257629041645-13911-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 13:47:37.446861 20818 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 13:47:37.449354 20818 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 13:47:37.449487 20818 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 13:47:37.449740 20818 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 13:47:37.450044 20818 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 13:47:37.450078 20818 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 13:47:37.450101 20818 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 13:47:37.450124 20818 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 13:47:37.457613 20818 rpc_server.cc:307] RPC server started. Bound to: 127.13.149.193:44033
I20260924 13:47:37.457647 21037 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.149.193:44033 every 8 connection(s)
I20260924 13:47:37.458108 20818 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790257629041645-13911-0/minicluster-data/ts-0/data/info.pb
I20260924 13:47:37.461797 13911 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJN9WRF/build/release/bin/kudu as pid 20818
I20260924 13:47:37.461964 13911 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJN9WRF/build/release/bin/kudu with pid 20270
Sep 24 13:47:37 dist-test-slave-sjl2 krb5kdc[20029](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257657, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.149.193@KRBTEST.COM for oryx/127.13.149.253@KRBTEST.COM
Sep 24 13:47:37 dist-test-slave-sjl2 krb5kdc[20029](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257657, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.149.193@KRBTEST.COM for oryx/127.13.149.252@KRBTEST.COM
Sep 24 13:47:37 dist-test-slave-sjl2 krb5kdc[20029](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257657, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.149.193@KRBTEST.COM for oryx/127.13.149.254@KRBTEST.COM
I20260924 13:47:37.469489 13911 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJN9WRF/build/release/bin/kudu
/tmp/dist-test-taskJN9WRF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790257629041645-13911-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790257629041645-13911-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790257629041645-13911-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790257629041645-13911-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.149.194:37849
--local_ip_for_outbound_sockets=127.13.149.194
--tserver_master_addrs=127.13.149.254:35859
--webserver_port=36543
--webserver_interface=127.13.149.194
--keytab_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790257629041645-13911-0/krb5kdc/oryx_127.13.149.194.keytab
--principal=oryx/127.13.149.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.149.212:44199
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.13.149.254:35859,127.13.149.253:38973
--tserver_master_addrs=127.13.149.254:35859,127.13.149.253:38973,127.13.149.252:35717 with env {KRB5CCNAME=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790257629041645-13911-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790257629041645-13911-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790257629041645-13911-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260924 13:47:37.471319 20710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:37.458917 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.149.193:51879 (local address 127.13.149.253:38973)
0924 13:47:37.458974 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0924 13:47:37.458977 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 13:47:37.459352 (+ 375us) server_negotiation.cc:407] Connection header received
0924 13:47:37.460639 (+ 1287us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:47:37.460641 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:47:37.460653 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:47:37.460682 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:47:37.461660 (+ 978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:37.462372 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:37.463721 (+ 1349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:37.464021 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:37.464060 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:47:37.467767 (+ 3707us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:47:37.467769 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:47:37.467769 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 13:47:37.467778 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:47:37.469683 (+ 1905us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:47:37.470104 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:47:37.470105 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:47:37.470106 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:47:37.470376 (+ 270us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:47:37.470640 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:47:37.470641 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:47:37.470641 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:47:37.470799 (+ 158us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:47:37.470852 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0924 13:47:37.471141 (+ 289us) server_negotiation.cc:299] Negotiation successful
0924 13:47:37.471226 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260924 13:47:37.471510 21042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:37.458935 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.253:38973 (local address 127.13.149.193:51879)
0924 13:47:37.459209 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0924 13:47:37.459263 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0924 13:47:37.460535 (+ 1272us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:47:37.460781 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:47:37.460787 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:47:37.461305 (+ 518us) client_negotiation.cc:189] Negotiated authn=SASL
0924 13:47:37.461528 (+ 223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:47:37.461535 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:47:37.462494 (+ 959us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:47:37.462495 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:47:37.463619 (+ 1124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:47:37.463621 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:47:37.464136 (+ 515us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:47:37.464137 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:47:37.464203 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:47:37.464708 (+ 505us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 13:47:37.464713 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 13:47:37.467677 (+ 2964us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 13:47:37.469769 (+ 2092us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 13:47:37.469770 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 13:47:37.469770 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 13:47:37.470021 (+ 251us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 13:47:37.470440 (+ 419us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 13:47:37.470441 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 13:47:37.470441 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 13:47:37.470570 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 13:47:37.470849 (+ 279us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 13:47:37.470857 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 13:47:37.471004 (+ 147us) client_negotiation.cc:769] Sending connection context
0924 13:47:37.471101 (+ 97us) client_negotiation.cc:240] Negotiation successful
0924 13:47:37.471274 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":205,"mutex_wait_us":39,"thread_start_us":81,"threads_started":1}
I20260924 13:47:37.472066 21040 heartbeater.cc:347] Connected to a master server at 127.13.149.253:38973
I20260924 13:47:37.472235 21040 heartbeater.cc:464] Registering TS with master...
I20260924 13:47:37.472487 21040 heartbeater.cc:511] Master 127.13.149.253:38973 requested a full tablet report, sending...
I20260924 13:47:37.472663 20113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:37.459634 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.149.193:55065 (local address 127.13.149.254:35859)
0924 13:47:37.459698 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0924 13:47:37.459700 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:47:37.459709 (+ 9us) server_negotiation.cc:407] Connection header received
0924 13:47:37.460762 (+ 1053us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:47:37.460763 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:47:37.460774 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:47:37.460803 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:47:37.465455 (+ 4652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:37.465951 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:37.467431 (+ 1480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:37.467686 (+ 255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:37.467751 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:47:37.469916 (+ 2165us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:47:37.469919 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:47:37.469919 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 13:47:37.469928 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:47:37.471468 (+ 1540us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:47:37.471867 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:47:37.471869 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:47:37.471869 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:47:37.471934 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:47:37.472110 (+ 176us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:47:37.472111 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:47:37.472112 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:47:37.472262 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:47:37.472338 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0924 13:47:37.472484 (+ 146us) server_negotiation.cc:299] Negotiation successful
0924 13:47:37.472571 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260924 13:47:37.472704 21044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:37.459850 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.252:35717 (local address 127.13.149.193:39903)
0924 13:47:37.460076 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0924 13:47:37.460087 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0924 13:47:37.460542 (+ 455us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:47:37.460805 (+ 263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:47:37.460805 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:47:37.461305 (+ 500us) client_negotiation.cc:189] Negotiated authn=SASL
0924 13:47:37.461528 (+ 223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:47:37.461535 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:47:37.462578 (+ 1043us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:47:37.462579 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:47:37.463482 (+ 903us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:47:37.463484 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:47:37.463863 (+ 379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:47:37.463864 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:47:37.463999 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:47:37.464508 (+ 509us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 13:47:37.464518 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0924 13:47:37.466943 (+ 2425us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 13:47:37.469302 (+ 2359us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 13:47:37.469314 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 13:47:37.469317 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0924 13:47:37.470021 (+ 704us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 13:47:37.470254 (+ 233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 13:47:37.470255 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 13:47:37.470256 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 13:47:37.470676 (+ 420us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 13:47:37.472330 (+ 1654us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 13:47:37.472331 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 13:47:37.472377 (+ 46us) client_negotiation.cc:769] Sending connection context
0924 13:47:37.472448 (+ 71us) client_negotiation.cc:240] Negotiation successful
0924 13:47:37.472586 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":209,"mutex_wait_us":89,"thread_start_us":61,"threads_started":1}
I20260924 13:47:37.473031 21045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:37.459932 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.149.193:39903 (local address 127.13.149.252:35717)
0924 13:47:37.460040 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0924 13:47:37.460042 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:47:37.460146 (+ 104us) server_negotiation.cc:407] Connection header received
0924 13:47:37.460705 (+ 559us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:47:37.460706 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:47:37.460720 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:47:37.460796 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:47:37.461645 (+ 849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:37.462498 (+ 853us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:37.463574 (+ 1076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:37.463791 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:37.463850 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:47:37.467054 (+ 3204us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:47:37.467056 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:47:37.467056 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 13:47:37.467067 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:47:37.469206 (+ 2139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:47:37.470104 (+ 898us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:47:37.470105 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:47:37.470106 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:47:37.470185 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:47:37.470775 (+ 590us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:47:37.472114 (+ 1339us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:47:37.472115 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:47:37.472245 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:47:37.472297 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0924 13:47:37.472802 (+ 505us) server_negotiation.cc:299] Negotiation successful
0924 13:47:37.472904 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87,"thread_start_us":62,"threads_started":1}
I20260924 13:47:37.473145 21043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:37.459437 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:35859 (local address 127.13.149.193:55065)
0924 13:47:37.459591 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0924 13:47:37.459601 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 13:47:37.460550 (+ 949us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:47:37.460865 (+ 315us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:47:37.460865 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:47:37.461302 (+ 437us) client_negotiation.cc:189] Negotiated authn=SASL
0924 13:47:37.461528 (+ 226us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:47:37.461533 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:47:37.466567 (+ 5034us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:47:37.466569 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:47:37.467338 (+ 769us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:47:37.467340 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:47:37.467814 (+ 474us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:47:37.467815 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:47:37.467890 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:47:37.468305 (+ 415us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 13:47:37.468310 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 13:47:37.469776 (+ 1466us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 13:47:37.471587 (+ 1811us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 13:47:37.471588 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 13:47:37.471589 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 13:47:37.471782 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 13:47:37.472002 (+ 220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 13:47:37.472003 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 13:47:37.472003 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 13:47:37.472055 (+ 52us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 13:47:37.472347 (+ 292us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 13:47:37.472348 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 13:47:37.472378 (+ 30us) client_negotiation.cc:769] Sending connection context
0924 13:47:37.472452 (+ 74us) client_negotiation.cc:240] Negotiation successful
0924 13:47:37.472662 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"mutex_wait_us":95,"thread_start_us":88,"threads_started":1}
I20260924 13:47:37.473359 21038 heartbeater.cc:347] Connected to a master server at 127.13.149.252:35717
I20260924 13:47:37.473410 21038 heartbeater.cc:464] Registering TS with master...
I20260924 13:47:37.473446 21039 heartbeater.cc:347] Connected to a master server at 127.13.149.254:35859
I20260924 13:47:37.473484 21039 heartbeater.cc:464] Registering TS with master...
I20260924 13:47:37.473493 21038 heartbeater.cc:511] Master 127.13.149.252:35717 requested a full tablet report, sending...
I20260924 13:47:37.473531 21039 heartbeater.cc:511] Master 127.13.149.254:35859 requested a full tablet report, sending...
I20260924 13:47:37.473759 20671 ts_manager.cc:194] Registered new tserver with Master: b79e3cc74b9d4c54bd7e3bc4626d08a1 (127.13.149.193:44033)
I20260924 13:47:37.473814 20074 ts_manager.cc:194] Re-registered known tserver with Master: b79e3cc74b9d4c54bd7e3bc4626d08a1 (127.13.149.193:44033)
I20260924 13:47:37.473832 20859 ts_manager.cc:194] Registered new tserver with Master: b79e3cc74b9d4c54bd7e3bc4626d08a1 (127.13.149.193:44033)
I20260924 13:47:37.474283 20074 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.13.149.193@KRBTEST.COM'} at 127.13.149.193:55065
W20260924 13:47:37.555505 21046 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 13:47:37.555714 21046 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 13:47:37.555738 21046 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 13:47:37.557081 21046 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 13:47:37.557132 21046 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 13:47:37.557170 21046 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.149.194
I20260924 13:47:37.558691 21046 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.149.212:44199
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790257629041645-13911-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790257629041645-13911-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790257629041645-13911-0/krb5kdc/oryx_127.13.149.194.keytab
--principal=oryx/127.13.149.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.149.194:37849
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790257629041645-13911-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.149.194
--webserver_port=36543
--tserver_master_addrs=127.13.149.254:35859,127.13.149.253:38973,127.13.149.252:35717
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21046
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.149.194
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790257629041645-13911-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 12:29:37 UTC on ccycloud.quasar-iqzbxn.root.comops.site
build id 4670106
I20260924 13:47:37.558897 21046 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 13:47:37.559082 21046 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 13:47:37.564147 21051 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 13:47:37.564141 21052 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 13:47:37.564137 21054 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 13:47:37.564312 21046 server_base.cc:1034] running on GCE node
I20260924 13:47:37.564524 21046 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 13:47:37.564775 21046 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 13:47:37.565941 21046 hybrid_clock.cc:648] HybridClock initialized: now 1790257657565904 us; error 46 us; skew 500 ppm
Sep 24 13:47:37 dist-test-slave-sjl2 krb5kdc[20029](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257657, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.149.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 13:47:37.568463 21046 init.cc:377] Logged in from keytab as oryx/127.13.149.194@KRBTEST.COM (short username oryx)
I20260924 13:47:37.568955 21046 webserver.cc:460] Webserver started at http://127.13.149.194:36543/ using document root <none> and password file <none>
I20260924 13:47:37.569175 21046 fs_manager.cc:359] Metadata directory not provided
I20260924 13:47:37.569242 21046 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 13:47:37.570693 21046 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260924 13:47:37.571285 21061 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 13:47:37.571429 21046 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 13:47:37.571508 21046 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790257629041645-13911-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790257629041645-13911-0/minicluster-data/ts-1/wal
uuid: "bc2e044c3d834bebbf3dc17a2c4da709"
format_stamp: "Formatted at 2026-09-24 13:47:36 on dist-test-slave-sjl2"
server_key: "0ad31aa84c904d4f349dcc4b61bd47d0"
server_key_iv: "3721edf7eded99bb41528746a6dbb9e7"
server_key_version: "encryptionkey@0"
I20260924 13:47:37.571851 21046 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790257629041645-13911-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790257629041645-13911-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790257629041645-13911-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 13:47:37.598099 21046 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 13:47:37.600312 21046 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 13:47:37.600435 21046 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 13:47:37.600680 21046 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 13:47:37.601011 21046 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 13:47:37.601048 21046 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 13:47:37.601079 21046 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 13:47:37.601118 21046 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 13:47:37.606696 21046 rpc_server.cc:307] RPC server started. Bound to: 127.13.149.194:37849
I20260924 13:47:37.606769 21174 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.149.194:37849 every 8 connection(s)
I20260924 13:47:37.607095 21046 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790257629041645-13911-0/minicluster-data/ts-1/data/info.pb
Sep 24 13:47:37 dist-test-slave-sjl2 krb5kdc[20029](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257657, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.149.194@KRBTEST.COM for oryx/127.13.149.253@KRBTEST.COM
Sep 24 13:47:37 dist-test-slave-sjl2 krb5kdc[20029](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257657, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.149.194@KRBTEST.COM for oryx/127.13.149.254@KRBTEST.COM
Sep 24 13:47:37 dist-test-slave-sjl2 krb5kdc[20029](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257657, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.149.194@KRBTEST.COM for oryx/127.13.149.252@KRBTEST.COM
I20260924 13:47:37.617041 13911 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJN9WRF/build/release/bin/kudu as pid 21046
I20260924 13:47:37.617201 13911 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJN9WRF/build/release/bin/kudu with pid 20404
I20260924 13:47:37.619091 20113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:37.607891 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.149.194:55687 (local address 127.13.149.254:35859)
0924 13:47:37.607959 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0924 13:47:37.607961 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:47:37.608154 (+ 193us) server_negotiation.cc:407] Connection header received
0924 13:47:37.609334 (+ 1180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:47:37.609335 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:47:37.609348 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:47:37.609398 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:47:37.610060 (+ 662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:37.610849 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:37.612320 (+ 1471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:37.612520 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:37.612573 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:47:37.615576 (+ 3003us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:47:37.615577 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:47:37.615578 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 13:47:37.615588 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:47:37.617227 (+ 1639us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:47:37.617750 (+ 523us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:47:37.617752 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:47:37.617752 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:47:37.617821 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:47:37.618237 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:47:37.618239 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:47:37.618239 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:47:37.618410 (+ 171us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:47:37.618464 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0924 13:47:37.618901 (+ 437us) server_negotiation.cc:299] Negotiation successful
0924 13:47:37.618992 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260924 13:47:37.619349 20710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:37.608387 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.149.194:34779 (local address 127.13.149.253:38973)
0924 13:47:37.608429 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0924 13:47:37.608431 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:47:37.608485 (+ 54us) server_negotiation.cc:407] Connection header received
0924 13:47:37.609342 (+ 857us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:47:37.609343 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:47:37.609354 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:47:37.609385 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:47:37.610090 (+ 705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:37.610842 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:37.612274 (+ 1432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:37.612554 (+ 280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:37.612617 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:47:37.615204 (+ 2587us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:47:37.615205 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:47:37.615205 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 13:47:37.615213 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:47:37.616717 (+ 1504us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:47:37.618561 (+ 1844us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:47:37.618562 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:47:37.618562 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:47:37.618632 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:47:37.618819 (+ 187us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:47:37.618819 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:47:37.618820 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:47:37.618945 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:47:37.618993 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0924 13:47:37.619173 (+ 180us) server_negotiation.cc:299] Negotiation successful
0924 13:47:37.619262 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260924 13:47:37.619710 21045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:37.608230 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.149.194:48627 (local address 127.13.149.252:35717)
0924 13:47:37.608459 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0924 13:47:37.608461 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:47:37.608471 (+ 10us) server_negotiation.cc:407] Connection header received
0924 13:47:37.609342 (+ 871us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:47:37.609343 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:47:37.609358 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:47:37.609387 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:47:37.610940 (+ 1553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:37.611410 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:37.612206 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:37.612504 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:37.612571 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:47:37.616237 (+ 3666us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:47:37.616240 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:47:37.616240 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 13:47:37.616250 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:47:37.618477 (+ 2227us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:47:37.618856 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:47:37.618858 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:47:37.618858 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:47:37.618919 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:47:37.619116 (+ 197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:47:37.619117 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:47:37.619117 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:47:37.619252 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:47:37.619293 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0924 13:47:37.619535 (+ 242us) server_negotiation.cc:299] Negotiation successful
0924 13:47:37.619621 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":201}
I20260924 13:47:37.619926 21179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:37.607873 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:35859 (local address 127.13.149.194:55687)
0924 13:47:37.608028 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0924 13:47:37.608070 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0924 13:47:37.609226 (+ 1156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:47:37.609464 (+ 238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:47:37.609467 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:47:37.609766 (+ 299us) client_negotiation.cc:189] Negotiated authn=SASL
0924 13:47:37.609971 (+ 205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:47:37.609975 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:47:37.610959 (+ 984us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:47:37.610959 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:47:37.612221 (+ 1262us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:47:37.612223 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:47:37.612629 (+ 406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:47:37.612629 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:47:37.612741 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:47:37.613368 (+ 627us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 13:47:37.613379 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0924 13:47:37.615380 (+ 2001us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 13:47:37.617371 (+ 1991us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 13:47:37.617372 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 13:47:37.617372 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 13:47:37.617650 (+ 278us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 13:47:37.617904 (+ 254us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 13:47:37.617905 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 13:47:37.617905 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 13:47:37.618059 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 13:47:37.618515 (+ 456us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 13:47:37.618523 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 13:47:37.618737 (+ 214us) client_negotiation.cc:769] Sending connection context
0924 13:47:37.619277 (+ 540us) client_negotiation.cc:240] Negotiation successful
0924 13:47:37.619571 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"mutex_wait_us":60,"thread_start_us":85,"threads_started":1}
I20260924 13:47:37.620139 21180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:37.608160 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.252:35717 (local address 127.13.149.194:48627)
0924 13:47:37.608268 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0924 13:47:37.608277 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0924 13:47:37.609232 (+ 955us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:47:37.609466 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:47:37.609468 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:47:37.609840 (+ 372us) client_negotiation.cc:189] Negotiated authn=SASL
0924 13:47:37.609973 (+ 133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:47:37.609979 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:47:37.611487 (+ 1508us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:47:37.611488 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:47:37.612122 (+ 634us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:47:37.612124 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:47:37.612586 (+ 462us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:47:37.612587 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:47:37.612741 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:47:37.613372 (+ 631us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 13:47:37.613379 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0924 13:47:37.615828 (+ 2449us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 13:47:37.618556 (+ 2728us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 13:47:37.618557 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 13:47:37.618558 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 13:47:37.618769 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 13:47:37.618988 (+ 219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 13:47:37.618988 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 13:47:37.618989 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 13:47:37.619041 (+ 52us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 13:47:37.619386 (+ 345us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 13:47:37.619387 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 13:47:37.619427 (+ 40us) client_negotiation.cc:769] Sending connection context
0924 13:47:37.619791 (+ 364us) client_negotiation.cc:240] Negotiation successful
0924 13:47:37.620032 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":91,"mutex_wait_us":163,"thread_start_us":57,"threads_started":1}
I20260924 13:47:37.620159 21181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:37.608337 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.253:38973 (local address 127.13.149.194:34779)
0924 13:47:37.608443 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0924 13:47:37.608451 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0924 13:47:37.609233 (+ 782us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:47:37.609465 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:47:37.609468 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:47:37.609838 (+ 370us) client_negotiation.cc:189] Negotiated authn=SASL
0924 13:47:37.609973 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:47:37.609979 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:47:37.610958 (+ 979us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:47:37.610959 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:47:37.612192 (+ 1233us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:47:37.612195 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:47:37.612669 (+ 474us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:47:37.612671 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:47:37.612740 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:47:37.613428 (+ 688us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 13:47:37.613433 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 13:47:37.615044 (+ 1611us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 13:47:37.616791 (+ 1747us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 13:47:37.616800 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 13:47:37.616803 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0924 13:47:37.618446 (+ 1643us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 13:47:37.618700 (+ 254us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 13:47:37.618701 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 13:47:37.618701 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 13:47:37.618759 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 13:47:37.619023 (+ 264us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 13:47:37.619024 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 13:47:37.619067 (+ 43us) client_negotiation.cc:769] Sending connection context
0924 13:47:37.619688 (+ 621us) client_negotiation.cc:240] Negotiation successful
0924 13:47:37.619941 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":93,"mutex_wait_us":22,"thread_start_us":64,"threads_started":1}
I20260924 13:47:37.620555 21177 heartbeater.cc:347] Connected to a master server at 127.13.149.253:38973
I20260924 13:47:37.620680 21177 heartbeater.cc:464] Registering TS with master...
I20260924 13:47:37.620869 21177 heartbeater.cc:511] Master 127.13.149.253:38973 requested a full tablet report, sending...
I20260924 13:47:37.621052 21175 heartbeater.cc:347] Connected to a master server at 127.13.149.252:35717
I20260924 13:47:37.621083 21175 heartbeater.cc:464] Registering TS with master...
I20260924 13:47:37.621150 21175 heartbeater.cc:511] Master 127.13.149.252:35717 requested a full tablet report, sending...
I20260924 13:47:37.621436 20859 ts_manager.cc:194] Registered new tserver with Master: bc2e044c3d834bebbf3dc17a2c4da709 (127.13.149.194:37849)
I20260924 13:47:37.621986 21176 heartbeater.cc:347] Connected to a master server at 127.13.149.254:35859
I20260924 13:47:37.622035 21176 heartbeater.cc:464] Registering TS with master...
I20260924 13:47:37.622104 21176 heartbeater.cc:511] Master 127.13.149.254:35859 requested a full tablet report, sending...
I20260924 13:47:37.622340 20074 ts_manager.cc:194] Re-registered known tserver with Master: bc2e044c3d834bebbf3dc17a2c4da709 (127.13.149.194:37849)
I20260924 13:47:37.622377 20671 ts_manager.cc:194] Registered new tserver with Master: bc2e044c3d834bebbf3dc17a2c4da709 (127.13.149.194:37849)
I20260924 13:47:37.622776 20074 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.13.149.194@KRBTEST.COM'} at 127.13.149.194:55687
I20260924 13:47:37.623411 13911 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJN9WRF/build/release/bin/kudu
/tmp/dist-test-taskJN9WRF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790257629041645-13911-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790257629041645-13911-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790257629041645-13911-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790257629041645-13911-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.149.195:41851
--local_ip_for_outbound_sockets=127.13.149.195
--tserver_master_addrs=127.13.149.254:35859
--webserver_port=35691
--webserver_interface=127.13.149.195
--keytab_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790257629041645-13911-0/krb5kdc/oryx_127.13.149.195.keytab
--principal=oryx/127.13.149.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.149.212:44199
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.13.149.254:35859,127.13.149.253:38973
--tserver_master_addrs=127.13.149.254:35859,127.13.149.253:38973,127.13.149.252:35717 with env {KRB5CCNAME=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790257629041645-13911-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790257629041645-13911-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790257629041645-13911-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 13:47:37.704839 21182 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 13:47:37.705122 21182 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 13:47:37.705149 21182 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 13:47:37.706403 21182 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 13:47:37.706449 21182 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 13:47:37.706477 21182 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.149.195
I20260924 13:47:37.708053 21182 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.149.212:44199
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790257629041645-13911-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790257629041645-13911-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790257629041645-13911-0/krb5kdc/oryx_127.13.149.195.keytab
--principal=oryx/127.13.149.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.149.195:41851
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790257629041645-13911-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.149.195
--webserver_port=35691
--tserver_master_addrs=127.13.149.254:35859,127.13.149.253:38973,127.13.149.252:35717
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21182
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.149.195
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790257629041645-13911-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 12:29:37 UTC on ccycloud.quasar-iqzbxn.root.comops.site
build id 4670106
I20260924 13:47:37.708305 21182 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 13:47:37.708544 21182 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 13:47:37.713562 21187 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 13:47:37.713647 21188 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 13:47:37.713559 21190 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 13:47:37.713718 21182 server_base.cc:1034] running on GCE node
I20260924 13:47:37.713975 21182 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 13:47:37.714205 21182 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 13:47:37.715364 21182 hybrid_clock.cc:648] HybridClock initialized: now 1790257657715351 us; error 31 us; skew 500 ppm
Sep 24 13:47:37 dist-test-slave-sjl2 krb5kdc[20029](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257657, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.149.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 13:47:37.717911 21182 init.cc:377] Logged in from keytab as oryx/127.13.149.195@KRBTEST.COM (short username oryx)
I20260924 13:47:37.718359 21182 webserver.cc:460] Webserver started at http://127.13.149.195:35691/ using document root <none> and password file <none>
I20260924 13:47:37.718564 21182 fs_manager.cc:359] Metadata directory not provided
I20260924 13:47:37.718626 21182 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 13:47:37.719928 21182 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260924 13:47:37.720448 21197 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 13:47:37.720600 21182 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 13:47:37.720721 21182 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790257629041645-13911-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790257629041645-13911-0/minicluster-data/ts-2/wal
uuid: "b056fa6235384b0ebb8123a47aaa399e"
format_stamp: "Formatted at 2026-09-24 13:47:36 on dist-test-slave-sjl2"
server_key: "5b0f0937ec2bc45d056a04d8ac78116a"
server_key_iv: "e576006ff804e75656e954ec3c27f446"
server_key_version: "encryptionkey@0"
I20260924 13:47:37.721115 21182 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790257629041645-13911-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790257629041645-13911-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790257629041645-13911-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 13:47:37.738545 21182 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 13:47:37.740656 21182 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 13:47:37.740803 21182 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 13:47:37.741088 21182 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 13:47:37.741415 21182 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 13:47:37.741447 21182 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 13:47:37.741478 21182 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 13:47:37.741503 21182 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 13:47:37.747133 21182 rpc_server.cc:307] RPC server started. Bound to: 127.13.149.195:41851
I20260924 13:47:37.747184 21310 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.149.195:41851 every 8 connection(s)
I20260924 13:47:37.747539 21182 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790257629041645-13911-0/minicluster-data/ts-2/data/info.pb
I20260924 13:47:37.748119 13911 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJN9WRF/build/release/bin/kudu as pid 21182
Sep 24 13:47:37 dist-test-slave-sjl2 krb5kdc[20029](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257657, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.149.195@KRBTEST.COM for oryx/127.13.149.254@KRBTEST.COM
Sep 24 13:47:37 dist-test-slave-sjl2 krb5kdc[20029](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257657, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.149.195@KRBTEST.COM for oryx/127.13.149.252@KRBTEST.COM
Sep 24 13:47:37 dist-test-slave-sjl2 krb5kdc[20029](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257657, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.149.195@KRBTEST.COM for oryx/127.13.149.253@KRBTEST.COM
I20260924 13:47:37.758482 21325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:37.749609 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53928 (local address 127.13.149.252:35717)
0924 13:47:37.749742 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0924 13:47:37.749744 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:47:37.750543 (+ 799us) server_negotiation.cc:407] Connection header received
0924 13:47:37.750568 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:47:37.750569 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:47:37.750582 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:47:37.750637 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:47:37.751718 (+ 1081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:37.752372 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:37.753406 (+ 1034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:37.753725 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:37.753781 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:47:37.755355 (+ 1574us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:47:37.755357 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:47:37.755357 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 13:47:37.755365 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:47:37.757038 (+ 1673us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:47:37.757480 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:47:37.757481 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:47:37.757481 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:47:37.757539 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:47:37.757744 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:47:37.757745 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:47:37.757745 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:47:37.757889 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:47:37.757995 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0924 13:47:37.758280 (+ 285us) server_negotiation.cc:299] Negotiation successful
0924 13:47:37.758373 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":114,"thread_start_us":66,"threads_started":1}
I20260924 13:47:37.761983 21316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:37.748664 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.252:35717 (local address 127.13.149.195:58755)
0924 13:47:37.749060 (+ 396us) negotiation.cc:107] Waiting for socket to connect
0924 13:47:37.749068 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0924 13:47:37.750107 (+ 1039us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:47:37.750317 (+ 210us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:47:37.750318 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:47:37.750880 (+ 562us) client_negotiation.cc:189] Negotiated authn=SASL
0924 13:47:37.751238 (+ 358us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:47:37.751246 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:47:37.752624 (+ 1378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:47:37.752625 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:47:37.753723 (+ 1098us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:47:37.753725 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:47:37.754257 (+ 532us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:47:37.754258 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:47:37.754412 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:47:37.755511 (+ 1099us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 13:47:37.755516 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 13:47:37.758079 (+ 2563us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 13:47:37.759923 (+ 1844us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 13:47:37.759937 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 13:47:37.759940 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0924 13:47:37.760296 (+ 356us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 13:47:37.760756 (+ 460us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 13:47:37.760757 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 13:47:37.760757 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 13:47:37.760939 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 13:47:37.761249 (+ 310us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 13:47:37.761258 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 13:47:37.761450 (+ 192us) client_negotiation.cc:769] Sending connection context
0924 13:47:37.761550 (+ 100us) client_negotiation.cc:240] Negotiation successful
0924 13:47:37.761737 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":381,"mutex_wait_us":10,"thread_start_us":92,"threads_started":1}
I20260924 13:47:37.762210 20113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:37.748379 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.149.195:49455 (local address 127.13.149.254:35859)
0924 13:47:37.748584 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0924 13:47:37.748587 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 13:47:37.748986 (+ 399us) server_negotiation.cc:407] Connection header received
0924 13:47:37.750258 (+ 1272us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:47:37.750260 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:47:37.750270 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:47:37.750317 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:47:37.752374 (+ 2057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:37.753153 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:37.754110 (+ 957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:37.754382 (+ 272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:37.754440 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:47:37.758967 (+ 4527us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:47:37.758969 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:47:37.758969 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 13:47:37.758978 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:47:37.760890 (+ 1912us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:47:37.761289 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:47:37.761290 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:47:37.761290 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:47:37.761347 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:47:37.761515 (+ 168us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:47:37.761515 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:47:37.761516 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:47:37.761633 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:47:37.761700 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0924 13:47:37.762040 (+ 340us) server_negotiation.cc:299] Negotiation successful
0924 13:47:37.762126 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":176}
I20260924 13:47:37.762367 21045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:37.748959 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.149.195:58755 (local address 127.13.149.252:35717)
0924 13:47:37.749211 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0924 13:47:37.749213 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:47:37.749221 (+ 8us) server_negotiation.cc:407] Connection header received
0924 13:47:37.750196 (+ 975us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:47:37.750197 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:47:37.750208 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:47:37.750253 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:47:37.751364 (+ 1111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:37.752140 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:37.753817 (+ 1677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:37.754123 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:37.754194 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:47:37.758182 (+ 3988us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:47:37.758183 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:47:37.758183 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 13:47:37.758189 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:47:37.759827 (+ 1638us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:47:37.760620 (+ 793us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:47:37.760622 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:47:37.760622 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:47:37.760682 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:47:37.761006 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:47:37.761007 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:47:37.761007 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:47:37.761158 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:47:37.761224 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0924 13:47:37.762210 (+ 986us) server_negotiation.cc:299] Negotiation successful
0924 13:47:37.762290 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":225}
I20260924 13:47:37.761989 21317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:37.748329 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:35859 (local address 127.13.149.195:49455)
0924 13:47:37.748852 (+ 523us) negotiation.cc:107] Waiting for socket to connect
0924 13:47:37.748862 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 13:47:37.749973 (+ 1111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:47:37.750385 (+ 412us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:47:37.750386 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:47:37.752225 (+ 1839us) client_negotiation.cc:189] Negotiated authn=SASL
0924 13:47:37.752292 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:47:37.752294 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:47:37.753240 (+ 946us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:47:37.753241 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:47:37.754008 (+ 767us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:47:37.754011 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:47:37.754492 (+ 481us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:47:37.754493 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:47:37.754554 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:47:37.755040 (+ 486us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 13:47:37.755051 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0924 13:47:37.758619 (+ 3568us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 13:47:37.761023 (+ 2404us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 13:47:37.761024 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 13:47:37.761024 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 13:47:37.761218 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 13:47:37.761405 (+ 187us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 13:47:37.761405 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 13:47:37.761406 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 13:47:37.761455 (+ 49us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 13:47:37.761726 (+ 271us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 13:47:37.761726 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 13:47:37.761763 (+ 37us) client_negotiation.cc:769] Sending connection context
0924 13:47:37.761819 (+ 56us) client_negotiation.cc:240] Negotiation successful
0924 13:47:37.761905 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":481,"mutex_wait_us":3,"thread_start_us":93,"threads_started":1}
I20260924 13:47:37.763223 21312 heartbeater.cc:347] Connected to a master server at 127.13.149.254:35859
I20260924 13:47:37.763230 21311 heartbeater.cc:347] Connected to a master server at 127.13.149.252:35717
I20260924 13:47:37.763374 21311 heartbeater.cc:464] Registering TS with master...
I20260924 13:47:37.763374 21312 heartbeater.cc:464] Registering TS with master...
I20260924 13:47:37.763592 21312 heartbeater.cc:511] Master 127.13.149.254:35859 requested a full tablet report, sending...
I20260924 13:47:37.763592 21311 heartbeater.cc:511] Master 127.13.149.252:35717 requested a full tablet report, sending...
I20260924 13:47:37.764063 20074 ts_manager.cc:194] Re-registered known tserver with Master: b056fa6235384b0ebb8123a47aaa399e (127.13.149.195:41851)
I20260924 13:47:37.764129 20859 ts_manager.cc:194] Registered new tserver with Master: b056fa6235384b0ebb8123a47aaa399e (127.13.149.195:41851)
I20260924 13:47:37.764289 21327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:37.750157 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37372 (local address 127.13.149.253:38973)
0924 13:47:37.750703 (+ 546us) server_negotiation.cc:206] Beginning negotiation
0924 13:47:37.750705 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:47:37.750711 (+ 6us) server_negotiation.cc:407] Connection header received
0924 13:47:37.750731 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:47:37.750732 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:47:37.750743 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:47:37.751522 (+ 779us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:47:37.751610 (+ 88us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:37.752316 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:37.756055 (+ 3739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:37.756483 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:37.756576 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:47:37.757867 (+ 1291us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:47:37.757868 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:47:37.757869 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 13:47:37.757876 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:47:37.761850 (+ 3974us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:47:37.762529 (+ 679us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:47:37.762530 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:47:37.762531 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:47:37.762594 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:47:37.762814 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:47:37.762815 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:47:37.762815 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:47:37.762958 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:47:37.764067 (+ 1109us) server_negotiation.cc:1035] Waiting for connection context
0924 13:47:37.764104 (+ 37us) server_negotiation.cc:299] Negotiation successful
0924 13:47:37.764194 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":527,"thread_start_us":61,"threads_started":1}
I20260924 13:47:37.764530 20074 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.13.149.195@KRBTEST.COM'} at 127.13.149.195:49455
I20260924 13:47:37.764837 20710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:37.748635 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.149.195:39997 (local address 127.13.149.253:38973)
0924 13:47:37.748860 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0924 13:47:37.748862 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:47:37.748872 (+ 10us) server_negotiation.cc:407] Connection header received
0924 13:47:37.750107 (+ 1235us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:47:37.750108 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:47:37.750120 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:47:37.750166 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:47:37.751365 (+ 1199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:37.752216 (+ 851us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:37.753823 (+ 1607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:37.754453 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:37.754496 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:47:37.759579 (+ 5083us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:47:37.759581 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:47:37.759581 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 13:47:37.759590 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:47:37.761776 (+ 2186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:47:37.763051 (+ 1275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:47:37.763052 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:47:37.763052 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:47:37.763100 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:47:37.763356 (+ 256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:47:37.763357 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:47:37.763357 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:47:37.763477 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:47:37.763523 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0924 13:47:37.764662 (+ 1139us) server_negotiation.cc:299] Negotiation successful
0924 13:47:37.764740 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":29,"server-negotiator.queue_time_us":196}
I20260924 13:47:37.764858 21323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:37.749491 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34344 (local address 127.13.149.254:35859)
0924 13:47:37.750150 (+ 659us) server_negotiation.cc:206] Beginning negotiation
0924 13:47:37.750152 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:47:37.750159 (+ 7us) server_negotiation.cc:407] Connection header received
0924 13:47:37.750182 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:47:37.750183 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:47:37.750195 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:47:37.754139 (+ 3944us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:47:37.754175 (+ 36us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:37.758383 (+ 4208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:37.759673 (+ 1290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:37.759995 (+ 322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:37.760361 (+ 366us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:47:37.761740 (+ 1379us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:47:37.761742 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:47:37.761742 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 13:47:37.761750 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:47:37.763425 (+ 1675us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:47:37.763783 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:47:37.763784 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:47:37.763784 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:47:37.763839 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:47:37.764034 (+ 195us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:47:37.764035 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:47:37.764035 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:47:37.764154 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:47:37.764635 (+ 481us) server_negotiation.cc:1035] Waiting for connection context
0924 13:47:37.764672 (+ 37us) server_negotiation.cc:299] Negotiation successful
0924 13:47:37.764747 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":631,"thread_start_us":73,"threads_started":1}
I20260924 13:47:37.765200 21315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:37.748329 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.253:38973 (local address 127.13.149.195:39997)
0924 13:47:37.748551 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0924 13:47:37.748596 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0924 13:47:37.749965 (+ 1369us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:47:37.750312 (+ 347us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:47:37.750317 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:47:37.750814 (+ 497us) client_negotiation.cc:189] Negotiated authn=SASL
0924 13:47:37.751239 (+ 425us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:47:37.751247 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:47:37.752366 (+ 1119us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:47:37.752367 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:47:37.753723 (+ 1356us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:47:37.753725 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:47:37.754574 (+ 849us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:47:37.754575 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:47:37.755053 (+ 478us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:47:37.755417 (+ 364us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 13:47:37.755421 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0924 13:47:37.759435 (+ 4014us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 13:47:37.761918 (+ 2483us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 13:47:37.761919 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 13:47:37.761919 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 13:47:37.762121 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 13:47:37.763165 (+ 1044us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 13:47:37.763166 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 13:47:37.763166 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 13:47:37.763280 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 13:47:37.763868 (+ 588us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 13:47:37.763869 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 13:47:37.763914 (+ 45us) client_negotiation.cc:769] Sending connection context
0924 13:47:37.764969 (+ 1055us) client_negotiation.cc:240] Negotiation successful
0924 13:47:37.765091 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":176,"mutex_wait_us":27,"thread_start_us":131,"threads_started":1}
I20260924 13:47:37.765509 21313 heartbeater.cc:347] Connected to a master server at 127.13.149.253:38973
I20260924 13:47:37.765554 21313 heartbeater.cc:464] Registering TS with master...
I20260924 13:47:37.765612 21313 heartbeater.cc:511] Master 127.13.149.253:38973 requested a full tablet report, sending...
I20260924 13:47:37.765926 20671 ts_manager.cc:194] Registered new tserver with Master: b056fa6235384b0ebb8123a47aaa399e (127.13.149.195:41851)
I20260924 13:47:37.766438 20074 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34344:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 13:47:37.768056 20074 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 13:47:37.776142 21335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:37.772264 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47680 (local address 127.13.149.194:37849)
0924 13:47:37.772664 (+ 400us) server_negotiation.cc:206] Beginning negotiation
0924 13:47:37.772668 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 13:47:37.772678 (+ 10us) server_negotiation.cc:407] Connection header received
0924 13:47:37.772715 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:47:37.772717 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:47:37.772732 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:47:37.772800 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 13:47:37.772904 (+ 104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:37.773999 (+ 1095us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:37.775237 (+ 1238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:37.775798 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:37.775874 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:47:37.775940 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0924 13:47:37.776005 (+ 65us) server_negotiation.cc:299] Negotiation successful
0924 13:47:37.776046 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":269,"thread_start_us":72,"threads_started":1}
I20260924 13:47:37.776224 21332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:37.772025 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.195:41851 (local address 127.0.0.1:52376)
0924 13:47:37.772185 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0924 13:47:37.772194 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0924 13:47:37.772226 (+ 32us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:47:37.773047 (+ 821us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:47:37.773048 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:47:37.773050 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 13:47:37.773086 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:47:37.773087 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:47:37.774135 (+ 1048us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:47:37.774136 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:47:37.775011 (+ 875us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:47:37.775014 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:47:37.775952 (+ 938us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:47:37.775952 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:47:37.776022 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:47:37.776033 (+ 11us) client_negotiation.cc:769] Sending connection context
0924 13:47:37.776058 (+ 25us) client_negotiation.cc:240] Negotiation successful
0924 13:47:37.776102 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"mutex_wait_us":7,"thread_start_us":85,"threads_started":1}
I20260924 13:47:37.776224 20822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:37.771977 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.193:44033 (local address 127.0.0.1:57682)
0924 13:47:37.772037 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0924 13:47:37.772049 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0924 13:47:37.772097 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:47:37.772447 (+ 350us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:47:37.772449 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:47:37.772452 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 13:47:37.772514 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:47:37.772516 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:47:37.773808 (+ 1292us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:47:37.773810 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:47:37.775027 (+ 1217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:47:37.775030 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:47:37.775950 (+ 920us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:47:37.775951 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:47:37.776022 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:47:37.776028 (+ 6us) client_negotiation.cc:769] Sending connection context
0924 13:47:37.776058 (+ 30us) client_negotiation.cc:240] Negotiation successful
0924 13:47:37.776102 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31,"mutex_wait_us":2}
I20260924 13:47:37.776458 20631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:37.771992 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.194:37849 (local address 127.0.0.1:47680)
0924 13:47:37.772133 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0924 13:47:37.772144 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0924 13:47:37.772185 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:47:37.772808 (+ 623us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:47:37.772809 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:47:37.772810 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 13:47:37.772840 (+ 30us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:47:37.772841 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:47:37.774097 (+ 1256us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:47:37.774098 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:47:37.775156 (+ 1058us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:47:37.775158 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:47:37.775870 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:47:37.775871 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:47:37.775930 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:47:37.775936 (+ 6us) client_negotiation.cc:769] Sending connection context
0924 13:47:37.775988 (+ 52us) client_negotiation.cc:240] Negotiation successful
0924 13:47:37.776044 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"mutex_wait_us":1}
I20260924 13:47:37.776603 21333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:37.772072 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57682 (local address 127.13.149.193:44033)
0924 13:47:37.772289 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0924 13:47:37.772294 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 13:47:37.772304 (+ 10us) server_negotiation.cc:407] Connection header received
0924 13:47:37.772344 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:47:37.772346 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:47:37.772362 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:47:37.772444 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 13:47:37.772579 (+ 135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:37.773717 (+ 1138us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:37.775118 (+ 1401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:37.775846 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:37.776184 (+ 338us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:47:37.776451 (+ 267us) server_negotiation.cc:1035] Waiting for connection context
0924 13:47:37.776479 (+ 28us) server_negotiation.cc:299] Negotiation successful
0924 13:47:37.776527 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":187,"thread_start_us":106,"threads_started":1}
I20260924 13:47:37.777223 20972 tablet_service.cc:1467] Processing CreateTablet for tablet 824436eb87bd4e178d5849b9aef0b3c7 (DEFAULT_TABLE table=test-table [id=33598122a06643b8971c8e5817166b0a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 13:47:37.777263 21109 tablet_service.cc:1467] Processing CreateTablet for tablet 824436eb87bd4e178d5849b9aef0b3c7 (DEFAULT_TABLE table=test-table [id=33598122a06643b8971c8e5817166b0a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 13:47:37.777593 20972 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 824436eb87bd4e178d5849b9aef0b3c7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 13:47:37.777603 21109 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 824436eb87bd4e178d5849b9aef0b3c7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 13:47:37.780512 21337 tablet_bootstrap.cc:492] T 824436eb87bd4e178d5849b9aef0b3c7 P bc2e044c3d834bebbf3dc17a2c4da709: Bootstrap starting.
I20260924 13:47:37.780689 21334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:37.772233 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52376 (local address 127.13.149.195:41851)
0924 13:47:37.772418 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0924 13:47:37.772423 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 13:47:37.772431 (+ 8us) server_negotiation.cc:407] Connection header received
0924 13:47:37.772895 (+ 464us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:47:37.772898 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:47:37.772912 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:47:37.773036 (+ 124us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 13:47:37.773145 (+ 109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:37.774052 (+ 907us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:37.775088 (+ 1036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:37.775849 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:37.776572 (+ 723us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:47:37.776651 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0924 13:47:37.776678 (+ 27us) server_negotiation.cc:299] Negotiation successful
0924 13:47:37.776725 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":150,"thread_start_us":111,"threads_started":1}
I20260924 13:47:37.781226 21337 tablet_bootstrap.cc:654] T 824436eb87bd4e178d5849b9aef0b3c7 P bc2e044c3d834bebbf3dc17a2c4da709: Neither blocks nor log segments found. Creating new log.
I20260924 13:47:37.781366 21245 tablet_service.cc:1467] Processing CreateTablet for tablet 824436eb87bd4e178d5849b9aef0b3c7 (DEFAULT_TABLE table=test-table [id=33598122a06643b8971c8e5817166b0a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 13:47:37.781483 21337 log.cc:824] T 824436eb87bd4e178d5849b9aef0b3c7 P bc2e044c3d834bebbf3dc17a2c4da709: Log is configured to *not* fsync() on all Append() calls
I20260924 13:47:37.781689 21245 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 824436eb87bd4e178d5849b9aef0b3c7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 13:47:37.781844 21336 tablet_bootstrap.cc:492] T 824436eb87bd4e178d5849b9aef0b3c7 P b79e3cc74b9d4c54bd7e3bc4626d08a1: Bootstrap starting.
I20260924 13:47:37.782274 21337 tablet_bootstrap.cc:492] T 824436eb87bd4e178d5849b9aef0b3c7 P bc2e044c3d834bebbf3dc17a2c4da709: No bootstrap required, opened a new log
I20260924 13:47:37.782347 21337 ts_tablet_manager.cc:1397] T 824436eb87bd4e178d5849b9aef0b3c7 P bc2e044c3d834bebbf3dc17a2c4da709: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260924 13:47:37.782512 21336 tablet_bootstrap.cc:654] T 824436eb87bd4e178d5849b9aef0b3c7 P b79e3cc74b9d4c54bd7e3bc4626d08a1: Neither blocks nor log segments found. Creating new log.
I20260924 13:47:37.782780 21336 log.cc:824] T 824436eb87bd4e178d5849b9aef0b3c7 P b79e3cc74b9d4c54bd7e3bc4626d08a1: Log is configured to *not* fsync() on all Append() calls
I20260924 13:47:37.783510 21336 tablet_bootstrap.cc:492] T 824436eb87bd4e178d5849b9aef0b3c7 P b79e3cc74b9d4c54bd7e3bc4626d08a1: No bootstrap required, opened a new log
I20260924 13:47:37.783571 21336 ts_tablet_manager.cc:1397] T 824436eb87bd4e178d5849b9aef0b3c7 P b79e3cc74b9d4c54bd7e3bc4626d08a1: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260924 13:47:37.784689 21337 raft_consensus.cc:348] T 824436eb87bd4e178d5849b9aef0b3c7 P bc2e044c3d834bebbf3dc17a2c4da709 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc2e044c3d834bebbf3dc17a2c4da709" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 37849 } } peers { permanent_uuid: "b79e3cc74b9d4c54bd7e3bc4626d08a1" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 44033 } } peers { permanent_uuid: "b056fa6235384b0ebb8123a47aaa399e" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 41851 } }
I20260924 13:47:37.784870 21337 raft_consensus.cc:374] T 824436eb87bd4e178d5849b9aef0b3c7 P bc2e044c3d834bebbf3dc17a2c4da709 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 13:47:37.784905 21337 raft_consensus.cc:729] T 824436eb87bd4e178d5849b9aef0b3c7 P bc2e044c3d834bebbf3dc17a2c4da709 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc2e044c3d834bebbf3dc17a2c4da709, State: Initialized, Role: FOLLOWER
I20260924 13:47:37.785081 21337 consensus_queue.cc:260] T 824436eb87bd4e178d5849b9aef0b3c7 P bc2e044c3d834bebbf3dc17a2c4da709 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc2e044c3d834bebbf3dc17a2c4da709" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 37849 } } peers { permanent_uuid: "b79e3cc74b9d4c54bd7e3bc4626d08a1" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 44033 } } peers { permanent_uuid: "b056fa6235384b0ebb8123a47aaa399e" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 41851 } }
I20260924 13:47:37.785324 21337 ts_tablet_manager.cc:1428] T 824436eb87bd4e178d5849b9aef0b3c7 P bc2e044c3d834bebbf3dc17a2c4da709: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260924 13:47:37.785611 21176 heartbeater.cc:503] Master 127.13.149.254:35859 was elected leader, sending a full tablet report...
I20260924 13:47:37.785617 21340 tablet_bootstrap.cc:492] T 824436eb87bd4e178d5849b9aef0b3c7 P b056fa6235384b0ebb8123a47aaa399e: Bootstrap starting.
I20260924 13:47:37.786347 21340 tablet_bootstrap.cc:654] T 824436eb87bd4e178d5849b9aef0b3c7 P b056fa6235384b0ebb8123a47aaa399e: Neither blocks nor log segments found. Creating new log.
I20260924 13:47:37.786545 21336 raft_consensus.cc:348] T 824436eb87bd4e178d5849b9aef0b3c7 P b79e3cc74b9d4c54bd7e3bc4626d08a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc2e044c3d834bebbf3dc17a2c4da709" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 37849 } } peers { permanent_uuid: "b79e3cc74b9d4c54bd7e3bc4626d08a1" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 44033 } } peers { permanent_uuid: "b056fa6235384b0ebb8123a47aaa399e" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 41851 } }
I20260924 13:47:37.786700 21340 log.cc:824] T 824436eb87bd4e178d5849b9aef0b3c7 P b056fa6235384b0ebb8123a47aaa399e: Log is configured to *not* fsync() on all Append() calls
I20260924 13:47:37.786715 21336 raft_consensus.cc:374] T 824436eb87bd4e178d5849b9aef0b3c7 P b79e3cc74b9d4c54bd7e3bc4626d08a1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 13:47:37.786751 21336 raft_consensus.cc:729] T 824436eb87bd4e178d5849b9aef0b3c7 P b79e3cc74b9d4c54bd7e3bc4626d08a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b79e3cc74b9d4c54bd7e3bc4626d08a1, State: Initialized, Role: FOLLOWER
I20260924 13:47:37.786866 21336 consensus_queue.cc:260] T 824436eb87bd4e178d5849b9aef0b3c7 P b79e3cc74b9d4c54bd7e3bc4626d08a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc2e044c3d834bebbf3dc17a2c4da709" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 37849 } } peers { permanent_uuid: "b79e3cc74b9d4c54bd7e3bc4626d08a1" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 44033 } } peers { permanent_uuid: "b056fa6235384b0ebb8123a47aaa399e" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 41851 } }
I20260924 13:47:37.787132 21336 ts_tablet_manager.cc:1428] T 824436eb87bd4e178d5849b9aef0b3c7 P b79e3cc74b9d4c54bd7e3bc4626d08a1: Time spent starting tablet: real 0.004s user 0.000s sys 0.003s
I20260924 13:47:37.787142 21039 heartbeater.cc:503] Master 127.13.149.254:35859 was elected leader, sending a full tablet report...
I20260924 13:47:37.787644 21340 tablet_bootstrap.cc:492] T 824436eb87bd4e178d5849b9aef0b3c7 P b056fa6235384b0ebb8123a47aaa399e: No bootstrap required, opened a new log
I20260924 13:47:37.787839 21340 ts_tablet_manager.cc:1397] T 824436eb87bd4e178d5849b9aef0b3c7 P b056fa6235384b0ebb8123a47aaa399e: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260924 13:47:37.789574 21340 raft_consensus.cc:348] T 824436eb87bd4e178d5849b9aef0b3c7 P b056fa6235384b0ebb8123a47aaa399e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc2e044c3d834bebbf3dc17a2c4da709" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 37849 } } peers { permanent_uuid: "b79e3cc74b9d4c54bd7e3bc4626d08a1" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 44033 } } peers { permanent_uuid: "b056fa6235384b0ebb8123a47aaa399e" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 41851 } }
I20260924 13:47:37.789765 21340 raft_consensus.cc:374] T 824436eb87bd4e178d5849b9aef0b3c7 P b056fa6235384b0ebb8123a47aaa399e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 13:47:37.789860 21340 raft_consensus.cc:729] T 824436eb87bd4e178d5849b9aef0b3c7 P b056fa6235384b0ebb8123a47aaa399e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b056fa6235384b0ebb8123a47aaa399e, State: Initialized, Role: FOLLOWER
I20260924 13:47:37.789986 21340 consensus_queue.cc:260] T 824436eb87bd4e178d5849b9aef0b3c7 P b056fa6235384b0ebb8123a47aaa399e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc2e044c3d834bebbf3dc17a2c4da709" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 37849 } } peers { permanent_uuid: "b79e3cc74b9d4c54bd7e3bc4626d08a1" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 44033 } } peers { permanent_uuid: "b056fa6235384b0ebb8123a47aaa399e" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 41851 } }
I20260924 13:47:37.790331 21340 ts_tablet_manager.cc:1428] T 824436eb87bd4e178d5849b9aef0b3c7 P b056fa6235384b0ebb8123a47aaa399e: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260924 13:47:37.790441 21312 heartbeater.cc:503] Master 127.13.149.254:35859 was elected leader, sending a full tablet report...
W20260924 13:47:37.857854 21178 tablet.cc:2369] T 824436eb87bd4e178d5849b9aef0b3c7 P bc2e044c3d834bebbf3dc17a2c4da709: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260924 13:47:37.892068 21344 raft_consensus.cc:482] T 824436eb87bd4e178d5849b9aef0b3c7 P b056fa6235384b0ebb8123a47aaa399e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 13:47:37.892202 21344 raft_consensus.cc:504] T 824436eb87bd4e178d5849b9aef0b3c7 P b056fa6235384b0ebb8123a47aaa399e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc2e044c3d834bebbf3dc17a2c4da709" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 37849 } } peers { permanent_uuid: "b79e3cc74b9d4c54bd7e3bc4626d08a1" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 44033 } } peers { permanent_uuid: "b056fa6235384b0ebb8123a47aaa399e" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 41851 } }
I20260924 13:47:37.892596 21344 leader_election.cc:288] T 824436eb87bd4e178d5849b9aef0b3c7 P b056fa6235384b0ebb8123a47aaa399e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bc2e044c3d834bebbf3dc17a2c4da709 (127.13.149.194:37849), b79e3cc74b9d4c54bd7e3bc4626d08a1 (127.13.149.193:44033)
I20260924 13:47:37.895318 21333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:37.892817 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.149.195:45905 (local address 127.13.149.193:44033)
0924 13:47:37.892871 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0924 13:47:37.892873 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:47:37.892961 (+ 88us) server_negotiation.cc:407] Connection header received
0924 13:47:37.893023 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:47:37.893024 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:47:37.893042 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:47:37.893089 (+ 47us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 13:47:37.893222 (+ 133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:37.893693 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:37.894604 (+ 911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:37.894993 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:37.895065 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:47:37.895074 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0924 13:47:37.895211 (+ 137us) server_negotiation.cc:299] Negotiation successful
0924 13:47:37.895250 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260924 13:47:37.895318 21335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:37.892849 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.149.195:56683 (local address 127.13.149.194:37849)
0924 13:47:37.892888 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0924 13:47:37.892890 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:47:37.892910 (+ 20us) server_negotiation.cc:407] Connection header received
0924 13:47:37.892966 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:47:37.892967 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:47:37.892975 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:47:37.893017 (+ 42us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 13:47:37.893166 (+ 149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:37.893658 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:37.894600 (+ 942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:37.894976 (+ 376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:37.895038 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:47:37.895048 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0924 13:47:37.895195 (+ 147us) server_negotiation.cc:299] Negotiation successful
0924 13:47:37.895249 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260924 13:47:37.895318 21315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:37.892758 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.194:37849 (local address 127.13.149.195:56683)
0924 13:47:37.892817 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0924 13:47:37.892829 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0924 13:47:37.892870 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:47:37.893035 (+ 165us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:47:37.893037 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:47:37.893038 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 13:47:37.893105 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:47:37.893107 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:47:37.893732 (+ 625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:47:37.893734 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:47:37.894504 (+ 770us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:47:37.894506 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:47:37.895049 (+ 543us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:47:37.895050 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:47:37.895131 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:47:37.895134 (+ 3us) client_negotiation.cc:769] Sending connection context
0924 13:47:37.895181 (+ 47us) client_negotiation.cc:240] Negotiation successful
0924 13:47:37.895225 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42}
I20260924 13:47:37.895325 21317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:37.892765 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.193:44033 (local address 127.13.149.195:45905)
0924 13:47:37.892894 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0924 13:47:37.892903 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0924 13:47:37.892975 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:47:37.893100 (+ 125us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:47:37.893101 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:47:37.893102 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 13:47:37.893169 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:47:37.893171 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:47:37.893776 (+ 605us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:47:37.893777 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:47:37.894504 (+ 727us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:47:37.894506 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:47:37.895069 (+ 563us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:47:37.895070 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:47:37.895151 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:47:37.895154 (+ 3us) client_negotiation.cc:769] Sending connection context
0924 13:47:37.895200 (+ 46us) client_negotiation.cc:240] Negotiation successful
0924 13:47:37.895245 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"mutex_wait_us":79}
I20260924 13:47:37.895686 21129 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "824436eb87bd4e178d5849b9aef0b3c7" candidate_uuid: "b056fa6235384b0ebb8123a47aaa399e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc2e044c3d834bebbf3dc17a2c4da709" is_pre_election: true
I20260924 13:47:37.895731 20992 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "824436eb87bd4e178d5849b9aef0b3c7" candidate_uuid: "b056fa6235384b0ebb8123a47aaa399e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b79e3cc74b9d4c54bd7e3bc4626d08a1" is_pre_election: true
I20260924 13:47:37.895849 21129 raft_consensus.cc:2454] T 824436eb87bd4e178d5849b9aef0b3c7 P bc2e044c3d834bebbf3dc17a2c4da709 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b056fa6235384b0ebb8123a47aaa399e in term 0.
I20260924 13:47:37.895892 20992 raft_consensus.cc:2454] T 824436eb87bd4e178d5849b9aef0b3c7 P b79e3cc74b9d4c54bd7e3bc4626d08a1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b056fa6235384b0ebb8123a47aaa399e in term 0.
I20260924 13:47:37.896147 21198 leader_election.cc:302] T 824436eb87bd4e178d5849b9aef0b3c7 P b056fa6235384b0ebb8123a47aaa399e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b056fa6235384b0ebb8123a47aaa399e, b79e3cc74b9d4c54bd7e3bc4626d08a1; no voters:
I20260924 13:47:37.896334 21344 raft_consensus.cc:2781] T 824436eb87bd4e178d5849b9aef0b3c7 P b056fa6235384b0ebb8123a47aaa399e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 13:47:37.896386 21344 raft_consensus.cc:482] T 824436eb87bd4e178d5849b9aef0b3c7 P b056fa6235384b0ebb8123a47aaa399e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 13:47:37.896420 21344 raft_consensus.cc:3037] T 824436eb87bd4e178d5849b9aef0b3c7 P b056fa6235384b0ebb8123a47aaa399e [term 0 FOLLOWER]: Advancing to term 1
I20260924 13:47:37.897352 21344 raft_consensus.cc:504] T 824436eb87bd4e178d5849b9aef0b3c7 P b056fa6235384b0ebb8123a47aaa399e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc2e044c3d834bebbf3dc17a2c4da709" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 37849 } } peers { permanent_uuid: "b79e3cc74b9d4c54bd7e3bc4626d08a1" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 44033 } } peers { permanent_uuid: "b056fa6235384b0ebb8123a47aaa399e" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 41851 } }
I20260924 13:47:37.897607 21344 leader_election.cc:288] T 824436eb87bd4e178d5849b9aef0b3c7 P b056fa6235384b0ebb8123a47aaa399e [CANDIDATE]: Term 1 election: Requested vote from peers bc2e044c3d834bebbf3dc17a2c4da709 (127.13.149.194:37849), b79e3cc74b9d4c54bd7e3bc4626d08a1 (127.13.149.193:44033)
I20260924 13:47:37.897706 20992 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "824436eb87bd4e178d5849b9aef0b3c7" candidate_uuid: "b056fa6235384b0ebb8123a47aaa399e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b79e3cc74b9d4c54bd7e3bc4626d08a1"
I20260924 13:47:37.897806 20992 raft_consensus.cc:3037] T 824436eb87bd4e178d5849b9aef0b3c7 P b79e3cc74b9d4c54bd7e3bc4626d08a1 [term 0 FOLLOWER]: Advancing to term 1
I20260924 13:47:37.897795 21129 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "824436eb87bd4e178d5849b9aef0b3c7" candidate_uuid: "b056fa6235384b0ebb8123a47aaa399e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc2e044c3d834bebbf3dc17a2c4da709"
I20260924 13:47:37.897902 21129 raft_consensus.cc:3037] T 824436eb87bd4e178d5849b9aef0b3c7 P bc2e044c3d834bebbf3dc17a2c4da709 [term 0 FOLLOWER]: Advancing to term 1
I20260924 13:47:37.898818 20992 raft_consensus.cc:2454] T 824436eb87bd4e178d5849b9aef0b3c7 P b79e3cc74b9d4c54bd7e3bc4626d08a1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b056fa6235384b0ebb8123a47aaa399e in term 1.
I20260924 13:47:37.898856 21129 raft_consensus.cc:2454] T 824436eb87bd4e178d5849b9aef0b3c7 P bc2e044c3d834bebbf3dc17a2c4da709 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b056fa6235384b0ebb8123a47aaa399e in term 1.
I20260924 13:47:37.898970 21198 leader_election.cc:302] T 824436eb87bd4e178d5849b9aef0b3c7 P b056fa6235384b0ebb8123a47aaa399e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b056fa6235384b0ebb8123a47aaa399e, b79e3cc74b9d4c54bd7e3bc4626d08a1; no voters:
I20260924 13:47:37.899426 21344 raft_consensus.cc:2781] T 824436eb87bd4e178d5849b9aef0b3c7 P b056fa6235384b0ebb8123a47aaa399e [term 1 FOLLOWER]: Leader election won for term 1
I20260924 13:47:37.899590 21344 raft_consensus.cc:686] T 824436eb87bd4e178d5849b9aef0b3c7 P b056fa6235384b0ebb8123a47aaa399e [term 1 LEADER]: Becoming Leader. State: Replica: b056fa6235384b0ebb8123a47aaa399e, State: Running, Role: LEADER
I20260924 13:47:37.899682 21344 consensus_queue.cc:237] T 824436eb87bd4e178d5849b9aef0b3c7 P b056fa6235384b0ebb8123a47aaa399e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc2e044c3d834bebbf3dc17a2c4da709" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 37849 } } peers { permanent_uuid: "b79e3cc74b9d4c54bd7e3bc4626d08a1" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 44033 } } peers { permanent_uuid: "b056fa6235384b0ebb8123a47aaa399e" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 41851 } }
I20260924 13:47:37.900492 20074 catalog_manager.cc:5495] T 824436eb87bd4e178d5849b9aef0b3c7 P b056fa6235384b0ebb8123a47aaa399e reported cstate change: term changed from 0 to 1, leader changed from <none> to b056fa6235384b0ebb8123a47aaa399e (127.13.149.195). New cstate: current_term: 1 leader_uuid: "b056fa6235384b0ebb8123a47aaa399e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc2e044c3d834bebbf3dc17a2c4da709" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 37849 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b79e3cc74b9d4c54bd7e3bc4626d08a1" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 44033 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b056fa6235384b0ebb8123a47aaa399e" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 41851 } health_report { overall_health: HEALTHY } } }
I20260924 13:47:37.923656 21334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:37.921176 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52392 (local address 127.13.149.195:41851)
0924 13:47:37.921230 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0924 13:47:37.921232 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:47:37.921253 (+ 21us) server_negotiation.cc:407] Connection header received
0924 13:47:37.921286 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:47:37.921287 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:47:37.921304 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:47:37.921354 (+ 50us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 13:47:37.921447 (+ 93us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:37.921948 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:37.922764 (+ 816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:37.922980 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:37.923041 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:47:37.923317 (+ 276us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 13:47:37.923428 (+ 111us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 13:47:37.923498 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0924 13:47:37.923530 (+ 32us) server_negotiation.cc:299] Negotiation successful
0924 13:47:37.923556 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260924 13:47:37.925593 20992 raft_consensus.cc:1261] T 824436eb87bd4e178d5849b9aef0b3c7 P b79e3cc74b9d4c54bd7e3bc4626d08a1 [term 1 FOLLOWER]: Refusing update from remote peer b056fa6235384b0ebb8123a47aaa399e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 13:47:37.925642 21129 raft_consensus.cc:1261] T 824436eb87bd4e178d5849b9aef0b3c7 P bc2e044c3d834bebbf3dc17a2c4da709 [term 1 FOLLOWER]: Refusing update from remote peer b056fa6235384b0ebb8123a47aaa399e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 13:47:37.925925 21344 consensus_queue.cc:1035] T 824436eb87bd4e178d5849b9aef0b3c7 P b056fa6235384b0ebb8123a47aaa399e [LEADER]: Connected to new peer: Peer: permanent_uuid: "b79e3cc74b9d4c54bd7e3bc4626d08a1" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 44033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 13:47:37.926077 21345 consensus_queue.cc:1035] T 824436eb87bd4e178d5849b9aef0b3c7 P b056fa6235384b0ebb8123a47aaa399e [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc2e044c3d834bebbf3dc17a2c4da709" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 37849 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 13:47:37.928807 21348 mvcc.cc:205] Tried to move back new op lower bound from 7332895366860328960 to 7332895366757814272. Current Snapshot: MvccSnapshot[applied={T|T < 7332895366860328960}]
I20260924 13:47:37.929591 21349 mvcc.cc:205] Tried to move back new op lower bound from 7332895366860328960 to 7332895366757814272. Current Snapshot: MvccSnapshot[applied={T|T < 7332895366860328960}]
I20260924 13:47:37.930783 21350 mvcc.cc:205] Tried to move back new op lower bound from 7332895366860328960 to 7332895366757814272. Current Snapshot: MvccSnapshot[applied={T|T < 7332895366860328960 or (T in {7332895366860328960})}]
I20260924 13:47:37.931334 20073 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34344:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 13:47:37.931422 20073 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34344:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 13:47:37.932844 20073 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9976d5567e674e309eecdd83a7ced13a: Sending DeleteTablet for 3 replicas of tablet 824436eb87bd4e178d5849b9aef0b3c7
I20260924 13:47:37.933177 21109 tablet_service.cc:1514] Processing DeleteTablet for tablet 824436eb87bd4e178d5849b9aef0b3c7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 13:47:37 UTC) from {username='oryx', principal='oryx/127.13.149.254@KRBTEST.COM'} at 127.0.0.1:47680
I20260924 13:47:37.933190 20972 tablet_service.cc:1514] Processing DeleteTablet for tablet 824436eb87bd4e178d5849b9aef0b3c7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 13:47:37 UTC) from {username='oryx', principal='oryx/127.13.149.254@KRBTEST.COM'} at 127.0.0.1:57682
I20260924 13:47:37.933482 21245 tablet_service.cc:1514] Processing DeleteTablet for tablet 824436eb87bd4e178d5849b9aef0b3c7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 13:47:37 UTC) from {username='oryx', principal='oryx/127.13.149.254@KRBTEST.COM'} at 127.0.0.1:52376
I20260924 13:47:37.933605 21358 tablet_replica.cc:330] T 824436eb87bd4e178d5849b9aef0b3c7 P bc2e044c3d834bebbf3dc17a2c4da709: stopping tablet replica
I20260924 13:47:37.933669 21360 tablet_replica.cc:330] T 824436eb87bd4e178d5849b9aef0b3c7 P b056fa6235384b0ebb8123a47aaa399e: stopping tablet replica
I20260924 13:47:37.933719 21358 raft_consensus.cc:2229] T 824436eb87bd4e178d5849b9aef0b3c7 P bc2e044c3d834bebbf3dc17a2c4da709 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 13:47:37.933746 21360 raft_consensus.cc:2229] T 824436eb87bd4e178d5849b9aef0b3c7 P b056fa6235384b0ebb8123a47aaa399e [term 1 LEADER]: Raft consensus shutting down.
I20260924 13:47:37.933825 21358 raft_consensus.cc:2258] T 824436eb87bd4e178d5849b9aef0b3c7 P bc2e044c3d834bebbf3dc17a2c4da709 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 13:47:37.933831 21360 raft_consensus.cc:2258] T 824436eb87bd4e178d5849b9aef0b3c7 P b056fa6235384b0ebb8123a47aaa399e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 13:47:37.934250 21358 ts_tablet_manager.cc:1905] T 824436eb87bd4e178d5849b9aef0b3c7 P bc2e044c3d834bebbf3dc17a2c4da709: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 13:47:37.934245 21360 ts_tablet_manager.cc:1905] T 824436eb87bd4e178d5849b9aef0b3c7 P b056fa6235384b0ebb8123a47aaa399e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 13:47:37.934978 13911 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJN9WRF/build/release/bin/kudu with pid 20042
I20260924 13:47:37.935839 21359 tablet_replica.cc:330] T 824436eb87bd4e178d5849b9aef0b3c7 P b79e3cc74b9d4c54bd7e3bc4626d08a1: stopping tablet replica
I20260924 13:47:37.935948 21359 raft_consensus.cc:2229] T 824436eb87bd4e178d5849b9aef0b3c7 P b79e3cc74b9d4c54bd7e3bc4626d08a1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 13:47:37.936064 21359 raft_consensus.cc:2258] T 824436eb87bd4e178d5849b9aef0b3c7 P b79e3cc74b9d4c54bd7e3bc4626d08a1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 13:47:37.936249 21360 ts_tablet_manager.cc:1918] T 824436eb87bd4e178d5849b9aef0b3c7 P b056fa6235384b0ebb8123a47aaa399e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 13:47:37.936318 21360 log.cc:1196] T 824436eb87bd4e178d5849b9aef0b3c7 P b056fa6235384b0ebb8123a47aaa399e: Deleting WAL directory at /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790257629041645-13911-0/minicluster-data/ts-2/wal/wals/824436eb87bd4e178d5849b9aef0b3c7
I20260924 13:47:37.936372 21358 ts_tablet_manager.cc:1918] T 824436eb87bd4e178d5849b9aef0b3c7 P bc2e044c3d834bebbf3dc17a2c4da709: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 13:47:37.936435 21358 log.cc:1196] T 824436eb87bd4e178d5849b9aef0b3c7 P bc2e044c3d834bebbf3dc17a2c4da709: Deleting WAL directory at /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790257629041645-13911-0/minicluster-data/ts-1/wal/wals/824436eb87bd4e178d5849b9aef0b3c7
I20260924 13:47:37.936535 21359 ts_tablet_manager.cc:1905] T 824436eb87bd4e178d5849b9aef0b3c7 P b79e3cc74b9d4c54bd7e3bc4626d08a1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 13:47:37.936563 21360 ts_tablet_manager.cc:1939] T 824436eb87bd4e178d5849b9aef0b3c7 P b056fa6235384b0ebb8123a47aaa399e: Deleting consensus metadata
I20260924 13:47:37.936694 21358 ts_tablet_manager.cc:1939] T 824436eb87bd4e178d5849b9aef0b3c7 P bc2e044c3d834bebbf3dc17a2c4da709: Deleting consensus metadata
I20260924 13:47:37.937948 21359 ts_tablet_manager.cc:1918] T 824436eb87bd4e178d5849b9aef0b3c7 P b79e3cc74b9d4c54bd7e3bc4626d08a1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 13:47:37.938000 21359 log.cc:1196] T 824436eb87bd4e178d5849b9aef0b3c7 P b79e3cc74b9d4c54bd7e3bc4626d08a1: Deleting WAL directory at /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790257629041645-13911-0/minicluster-data/ts-0/wal/wals/824436eb87bd4e178d5849b9aef0b3c7
I20260924 13:47:37.938166 21359 ts_tablet_manager.cc:1939] T 824436eb87bd4e178d5849b9aef0b3c7 P b79e3cc74b9d4c54bd7e3bc4626d08a1: Deleting consensus metadata
W20260924 13:47:37.941471 20928 connection.cc:535] server connection from 127.0.0.1:57682 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260924 13:47:37.941483 21200 connection.cc:535] server connection from 127.0.0.1:52376 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260924 13:47:37.941812 21063 connection.cc:535] server connection from 127.0.0.1:47680 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260924 13:47:37.950384 21045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:37.933959 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53934 (local address 127.13.149.252:35717)
0924 13:47:37.934002 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0924 13:47:37.934004 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:47:37.942206 (+ 8202us) server_negotiation.cc:407] Connection header received
0924 13:47:37.942273 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:47:37.942274 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:47:37.942288 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:47:37.942350 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:47:37.943086 (+ 736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:37.943860 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:37.944974 (+ 1114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:37.945266 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:37.945331 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:47:37.946877 (+ 1546us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:47:37.946878 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:47:37.946879 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 13:47:37.946886 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:47:37.948659 (+ 1773us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:47:37.949370 (+ 711us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:47:37.949371 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:47:37.949372 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:47:37.949432 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:47:37.949666 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:47:37.949667 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:47:37.949667 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:47:37.949922 (+ 255us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:47:37.950011 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0924 13:47:37.950203 (+ 192us) server_negotiation.cc:299] Negotiation successful
0924 13:47:37.950283 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260924 13:47:37.950516 21325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:37.943158 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53938 (local address 127.13.149.252:35717)
0924 13:47:37.943197 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0924 13:47:37.943199 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:47:37.943250 (+ 51us) server_negotiation.cc:407] Connection header received
0924 13:47:37.943326 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:47:37.943327 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:47:37.943339 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:47:37.943379 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:47:37.944142 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:37.944623 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:37.945558 (+ 935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:37.945751 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:37.945799 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:47:37.946944 (+ 1145us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:47:37.946946 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:47:37.946946 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 13:47:37.946953 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:47:37.948787 (+ 1834us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:47:37.949649 (+ 862us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:47:37.949651 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:47:37.949651 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:47:37.949724 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:47:37.949929 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:47:37.949930 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:47:37.949930 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:47:37.950058 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:47:37.950127 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0924 13:47:37.950335 (+ 208us) server_negotiation.cc:299] Negotiation successful
0924 13:47:37.950424 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260924 13:47:37.955361 21327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:37.949174 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37374 (local address 127.13.149.253:38973)
0924 13:47:37.949225 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0924 13:47:37.949228 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 13:47:37.949404 (+ 176us) server_negotiation.cc:407] Connection header received
0924 13:47:37.949449 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:47:37.949449 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:47:37.949462 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:47:37.949507 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:47:37.950052 (+ 545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:37.950782 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:37.951554 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:47:37.951828 (+ 274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:47:37.951890 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:47:37.953151 (+ 1261us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:47:37.953153 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:47:37.953153 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 13:47:37.953161 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:47:37.954348 (+ 1187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:47:37.954654 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:47:37.954655 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:47:37.954656 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:47:37.954711 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:47:37.954870 (+ 159us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:47:37.954871 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:47:37.954871 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:47:37.954987 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:47:37.955076 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0924 13:47:37.955179 (+ 103us) server_negotiation.cc:299] Negotiation successful
0924 13:47:37.955268 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260924 13:47:38.930461 21368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:38.930127 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:35859 (local address 127.13.149.195:43333)
0924 13:47:38.930309 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0924 13:47:38.930402 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.149.254:35859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":67,"threads_started":1}
W20260924 13:47:38.930737 21312 heartbeater.cc:644] Failed to heartbeat to 127.13.149.254:35859 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.13.149.254:35859: connect: Connection refused (error 111)
I20260924 13:47:38.930879 21369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:38.930614 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:35859 (local address 127.13.149.194:53323)
0924 13:47:38.930750 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0924 13:47:38.930835 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.149.254:35859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":50,"threads_started":1}
W20260924 13:47:38.931377 21176 heartbeater.cc:644] Failed to heartbeat to 127.13.149.254:35859 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.13.149.254:35859: connect: Connection refused (error 111)
I20260924 13:47:38.931413 21370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:38.931143 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:35859 (local address 127.13.149.193:35567)
0924 13:47:38.931309 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0924 13:47:38.931379 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.149.254:35859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":68,"threads_started":1}
W20260924 13:47:38.931576 21039 heartbeater.cc:644] Failed to heartbeat to 127.13.149.254:35859 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.13.149.254:35859: connect: Connection refused (error 111)
I20260924 13:47:39.433702 21371 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1d4faf110d7a41cc8a2113c5b74a298a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9976d5567e674e309eecdd83a7ced13a)
I20260924 13:47:39.433940 21371 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1d4faf110d7a41cc8a2113c5b74a298a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "9976d5567e674e309eecdd83a7ced13a" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 35859 } } peers { permanent_uuid: "1d4faf110d7a41cc8a2113c5b74a298a" member_type: VOTER last_known_addr { host: "127.13.149.253" port: 38973 } attrs { promote: false } } peers { permanent_uuid: "e42b5e67f6b6495080300913733a80db" member_type: NON_VOTER last_known_addr { host: "127.13.149.252" port: 35717 } attrs { promote: true } }
I20260924 13:47:39.434157 21371 leader_election.cc:288] T 00000000000000000000000000000000 P 1d4faf110d7a41cc8a2113c5b74a298a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9976d5567e674e309eecdd83a7ced13a (127.13.149.254:35859)
I20260924 13:47:39.434505 21372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:39.434320 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:35859 (local address 127.0.0.1:37466)
0924 13:47:39.434420 (+ 100us) negotiation.cc:107] Waiting for socket to connect
0924 13:47:39.434469 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.149.254:35859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82,"thread_start_us":51,"threads_started":1}
W20260924 13:47:39.434633 20658 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.13.149.254:35859: connect: Connection refused (error 111)
I20260924 13:47:39.435074 21372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:39.434984 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:35859 (local address 127.0.0.1:37478)
0924 13:47:39.435028 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0924 13:47:39.435055 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.149.254:35859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260924 13:47:39.435180 20658 leader_election.cc:334] T 00000000000000000000000000000000 P 1d4faf110d7a41cc8a2113c5b74a298a [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 9976d5567e674e309eecdd83a7ced13a (127.13.149.254:35859): Network error: Client connection negotiation failed: client connection to 127.13.149.254:35859: connect: Connection refused (error 111)
I20260924 13:47:39.435243 20658 leader_election.cc:302] T 00000000000000000000000000000000 P 1d4faf110d7a41cc8a2113c5b74a298a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 2 responses out of 2 voters: 1 yes votes; 1 no votes. yes voters: 1d4faf110d7a41cc8a2113c5b74a298a; no voters: 9976d5567e674e309eecdd83a7ced13a
I20260924 13:47:39.435385 21371 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 1d4faf110d7a41cc8a2113c5b74a298a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260924 13:47:39.931377 21374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:39.931134 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:35859 (local address 127.13.149.195:34885)
0924 13:47:39.931275 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0924 13:47:39.931318 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.149.254:35859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":61,"threads_started":1}
I20260924 13:47:39.931803 21375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:39.931627 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:35859 (local address 127.13.149.194:37343)
0924 13:47:39.931726 (+ 99us) negotiation.cc:107] Waiting for socket to connect
0924 13:47:39.931762 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.149.254:35859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":80,"thread_start_us":48,"threads_started":1}
I20260924 13:47:39.932039 21376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:39.931843 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:35859 (local address 127.13.149.193:51407)
0924 13:47:39.931969 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0924 13:47:39.932005 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.149.254:35859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":61,"threads_started":1}
I20260924 13:47:40.932266 21377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:40.931972 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:35859 (local address 127.13.149.195:34725)
0924 13:47:40.932161 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0924 13:47:40.932208 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.149.254:35859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":70,"threads_started":1}
I20260924 13:47:40.932556 21378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:40.932254 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:35859 (local address 127.13.149.194:48121)
0924 13:47:40.932458 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0924 13:47:40.932512 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.149.254:35859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":56,"threads_started":1}
I20260924 13:47:40.932751 21379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:40.932454 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:35859 (local address 127.13.149.193:34367)
0924 13:47:40.932681 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0924 13:47:40.932712 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.149.254:35859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":211,"thread_start_us":51,"threads_started":1}
I20260924 13:47:41.233676 21380 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1d4faf110d7a41cc8a2113c5b74a298a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9976d5567e674e309eecdd83a7ced13a)
I20260924 13:47:41.233794 21380 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1d4faf110d7a41cc8a2113c5b74a298a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "9976d5567e674e309eecdd83a7ced13a" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 35859 } } peers { permanent_uuid: "1d4faf110d7a41cc8a2113c5b74a298a" member_type: VOTER last_known_addr { host: "127.13.149.253" port: 38973 } attrs { promote: false } } peers { permanent_uuid: "e42b5e67f6b6495080300913733a80db" member_type: NON_VOTER last_known_addr { host: "127.13.149.252" port: 35717 } attrs { promote: true } }
I20260924 13:47:41.233976 21380 leader_election.cc:288] T 00000000000000000000000000000000 P 1d4faf110d7a41cc8a2113c5b74a298a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9976d5567e674e309eecdd83a7ced13a (127.13.149.254:35859)
I20260924 13:47:41.234346 21381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:41.234139 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:35859 (local address 127.0.0.1:37704)
0924 13:47:41.234266 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0924 13:47:41.234310 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.149.254:35859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":48,"threads_started":1}
I20260924 13:47:41.234678 21381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:41.234605 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:35859 (local address 127.0.0.1:37712)
0924 13:47:41.234641 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0924 13:47:41.234664 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.149.254:35859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
W20260924 13:47:41.234736 20658 leader_election.cc:334] T 00000000000000000000000000000000 P 1d4faf110d7a41cc8a2113c5b74a298a [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 9976d5567e674e309eecdd83a7ced13a (127.13.149.254:35859): Network error: Client connection negotiation failed: client connection to 127.13.149.254:35859: connect: Connection refused (error 111)
I20260924 13:47:41.234788 20658 leader_election.cc:302] T 00000000000000000000000000000000 P 1d4faf110d7a41cc8a2113c5b74a298a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 2 responses out of 2 voters: 1 yes votes; 1 no votes. yes voters: 1d4faf110d7a41cc8a2113c5b74a298a; no voters: 9976d5567e674e309eecdd83a7ced13a
I20260924 13:47:41.234859 21380 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 1d4faf110d7a41cc8a2113c5b74a298a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260924 13:47:41.933379 21384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:41.933157 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:35859 (local address 127.13.149.193:36515)
0924 13:47:41.933281 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0924 13:47:41.933322 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.149.254:35859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":54,"threads_started":1}
I20260924 13:47:41.933605 21383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:41.933077 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:35859 (local address 127.13.149.195:53621)
0924 13:47:41.933528 (+ 451us) negotiation.cc:107] Waiting for socket to connect
0924 13:47:41.933564 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.149.254:35859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":433,"thread_start_us":87,"threads_started":1}
I20260924 13:47:41.933677 21385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:41.933299 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:35859 (local address 127.13.149.194:43261)
0924 13:47:41.933559 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0924 13:47:41.933640 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.149.254:35859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":241,"thread_start_us":46,"threads_started":1}
I20260924 13:47:42.934296 21387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:42.933994 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:35859 (local address 127.13.149.195:34383)
0924 13:47:42.934196 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0924 13:47:42.934240 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.149.254:35859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":87,"threads_started":1}
I20260924 13:47:42.934296 21386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:42.934018 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:35859 (local address 127.13.149.193:55553)
0924 13:47:42.934186 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0924 13:47:42.934240 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.149.254:35859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":64,"threads_started":1}
I20260924 13:47:42.934412 21388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:42.934032 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:35859 (local address 127.13.149.194:41319)
0924 13:47:42.934339 (+ 307us) negotiation.cc:107] Waiting for socket to connect
0924 13:47:42.934375 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.149.254:35859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":289,"thread_start_us":53,"threads_started":1}
I20260924 13:47:43.697770 21389 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1d4faf110d7a41cc8a2113c5b74a298a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9976d5567e674e309eecdd83a7ced13a)
I20260924 13:47:43.697902 21389 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1d4faf110d7a41cc8a2113c5b74a298a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "9976d5567e674e309eecdd83a7ced13a" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 35859 } } peers { permanent_uuid: "1d4faf110d7a41cc8a2113c5b74a298a" member_type: VOTER last_known_addr { host: "127.13.149.253" port: 38973 } attrs { promote: false } } peers { permanent_uuid: "e42b5e67f6b6495080300913733a80db" member_type: NON_VOTER last_known_addr { host: "127.13.149.252" port: 35717 } attrs { promote: true } }
I20260924 13:47:43.698062 21389 leader_election.cc:288] T 00000000000000000000000000000000 P 1d4faf110d7a41cc8a2113c5b74a298a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9976d5567e674e309eecdd83a7ced13a (127.13.149.254:35859)
I20260924 13:47:43.698377 21390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:43.698204 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:35859 (local address 127.0.0.1:37940)
0924 13:47:43.698303 (+ 99us) negotiation.cc:107] Waiting for socket to connect
0924 13:47:43.698340 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.149.254:35859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":80,"thread_start_us":43,"threads_started":1}
I20260924 13:47:43.698853 21390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:43.698721 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:35859 (local address 127.0.0.1:37954)
0924 13:47:43.698786 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0924 13:47:43.698818 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.149.254:35859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260924 13:47:43.698936 20658 leader_election.cc:334] T 00000000000000000000000000000000 P 1d4faf110d7a41cc8a2113c5b74a298a [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 9976d5567e674e309eecdd83a7ced13a (127.13.149.254:35859): Network error: Client connection negotiation failed: client connection to 127.13.149.254:35859: connect: Connection refused (error 111)
I20260924 13:47:43.698997 20658 leader_election.cc:302] T 00000000000000000000000000000000 P 1d4faf110d7a41cc8a2113c5b74a298a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 2 responses out of 2 voters: 1 yes votes; 1 no votes. yes voters: 1d4faf110d7a41cc8a2113c5b74a298a; no voters: 9976d5567e674e309eecdd83a7ced13a
I20260924 13:47:43.699092 21389 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 1d4faf110d7a41cc8a2113c5b74a298a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260924 13:47:43.935065 21392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:43.934831 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:35859 (local address 127.13.149.194:58371)
0924 13:47:43.934959 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0924 13:47:43.935010 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.149.254:35859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":66,"threads_started":1}
I20260924 13:47:43.935199 21393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:43.934863 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:35859 (local address 127.13.149.195:45309)
0924 13:47:43.935114 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0924 13:47:43.935151 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.149.254:35859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":229,"thread_start_us":35,"threads_started":1}
I20260924 13:47:43.935290 21394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:43.935060 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:35859 (local address 127.13.149.193:59063)
0924 13:47:43.935223 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0924 13:47:43.935252 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.149.254:35859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":57,"threads_started":1}
I20260924 13:47:44.935839 21395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:44.935598 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:35859 (local address 127.13.149.193:58339)
0924 13:47:44.935719 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0924 13:47:44.935778 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.149.254:35859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":69,"threads_started":1}
I20260924 13:47:44.936254 21396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:44.935659 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:35859 (local address 127.13.149.195:51089)
0924 13:47:44.936171 (+ 512us) negotiation.cc:107] Waiting for socket to connect
0924 13:47:44.936209 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.149.254:35859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":495,"thread_start_us":50,"threads_started":1}
I20260924 13:47:44.936262 21397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:44.936034 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:35859 (local address 127.13.149.194:34359)
0924 13:47:44.936170 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0924 13:47:44.936211 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.149.254:35859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":49,"threads_started":1}
I20260924 13:47:45.936893 21399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:45.936571 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:35859 (local address 127.13.149.194:32897)
0924 13:47:45.936795 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0924 13:47:45.936830 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.149.254:35859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":206,"thread_start_us":45,"threads_started":1}
I20260924 13:47:45.937076 21400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:45.936700 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:35859 (local address 127.13.149.195:57321)
0924 13:47:45.937002 (+ 302us) negotiation.cc:107] Waiting for socket to connect
0924 13:47:45.937039 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.149.254:35859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":278,"thread_start_us":47,"threads_started":1}
I20260924 13:47:45.937214 21398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:45.936391 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:35859 (local address 127.13.149.193:48173)
0924 13:47:45.936594 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0924 13:47:45.936644 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.149.254:35859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":66,"threads_started":1}
I20260924 13:47:46.937829 21402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:46.937486 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:35859 (local address 127.13.149.195:47885)
0924 13:47:46.937727 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0924 13:47:46.937772 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.149.254:35859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":220,"thread_start_us":67,"threads_started":1}
I20260924 13:47:46.937940 21403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:46.937688 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:35859 (local address 127.13.149.193:52337)
0924 13:47:46.937850 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0924 13:47:46.937891 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.149.254:35859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":44,"threads_started":1}
I20260924 13:47:46.938090 21401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:46.937445 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:35859 (local address 127.13.149.194:50759)
0924 13:47:46.938017 (+ 572us) negotiation.cc:107] Waiting for socket to connect
0924 13:47:46.938051 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.149.254:35859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":554,"thread_start_us":61,"threads_started":1}
I20260924 13:47:47.200124 21404 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1d4faf110d7a41cc8a2113c5b74a298a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9976d5567e674e309eecdd83a7ced13a)
I20260924 13:47:47.200227 21404 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1d4faf110d7a41cc8a2113c5b74a298a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "9976d5567e674e309eecdd83a7ced13a" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 35859 } } peers { permanent_uuid: "1d4faf110d7a41cc8a2113c5b74a298a" member_type: VOTER last_known_addr { host: "127.13.149.253" port: 38973 } attrs { promote: false } } peers { permanent_uuid: "e42b5e67f6b6495080300913733a80db" member_type: NON_VOTER last_known_addr { host: "127.13.149.252" port: 35717 } attrs { promote: true } }
I20260924 13:47:47.200352 21404 leader_election.cc:288] T 00000000000000000000000000000000 P 1d4faf110d7a41cc8a2113c5b74a298a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9976d5567e674e309eecdd83a7ced13a (127.13.149.254:35859)
I20260924 13:47:47.200685 21405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:47.200506 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:35859 (local address 127.0.0.1:38252)
0924 13:47:47.200602 (+ 96us) negotiation.cc:107] Waiting for socket to connect
0924 13:47:47.200646 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.149.254:35859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78,"thread_start_us":46,"threads_started":1}
W20260924 13:47:47.200763 20658 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.13.149.254:35859: connect: Connection refused (error 111) [suppressed 2 similar messages]
I20260924 13:47:47.201155 21405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:47.201049 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:35859 (local address 127.0.0.1:38266)
0924 13:47:47.201102 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0924 13:47:47.201133 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.149.254:35859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260924 13:47:47.201227 20658 leader_election.cc:334] T 00000000000000000000000000000000 P 1d4faf110d7a41cc8a2113c5b74a298a [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 9976d5567e674e309eecdd83a7ced13a (127.13.149.254:35859): Network error: Client connection negotiation failed: client connection to 127.13.149.254:35859: connect: Connection refused (error 111)
I20260924 13:47:47.201267 20658 leader_election.cc:302] T 00000000000000000000000000000000 P 1d4faf110d7a41cc8a2113c5b74a298a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 2 responses out of 2 voters: 1 yes votes; 1 no votes. yes voters: 1d4faf110d7a41cc8a2113c5b74a298a; no voters: 9976d5567e674e309eecdd83a7ced13a
I20260924 13:47:47.201333 21404 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 1d4faf110d7a41cc8a2113c5b74a298a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260924 13:47:47.938920 21409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:47.938688 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:35859 (local address 127.13.149.195:35771)
0924 13:47:47.938835 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0924 13:47:47.938868 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.149.254:35859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":56,"threads_started":1}
I20260924 13:47:47.938920 21408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:47.938631 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:35859 (local address 127.13.149.194:60569)
0924 13:47:47.938829 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0924 13:47:47.938869 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.149.254:35859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":42,"threads_started":1}
I20260924 13:47:47.938923 21407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:47.938590 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:35859 (local address 127.13.149.193:51217)
0924 13:47:47.938827 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0924 13:47:47.938869 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.149.254:35859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":217,"thread_start_us":67,"threads_started":1}
I20260924 13:47:48.939594 21410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:48.939310 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:35859 (local address 127.13.149.193:56867)
0924 13:47:48.939485 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0924 13:47:48.939535 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.149.254:35859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":72,"threads_started":1}
I20260924 13:47:48.939723 21411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:48.939455 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:35859 (local address 127.13.149.195:43553)
0924 13:47:48.939590 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0924 13:47:48.939629 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.149.254:35859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":65,"threads_started":1}
I20260924 13:47:48.940013 21412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:48.939632 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:35859 (local address 127.13.149.194:41575)
0924 13:47:48.939875 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0924 13:47:48.939971 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.149.254:35859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":225,"thread_start_us":44,"threads_started":1}
I20260924 13:47:49.940429 21413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:49.940076 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:35859 (local address 127.13.149.193:55059)
0924 13:47:49.940331 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0924 13:47:49.940371 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.149.254:35859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":234,"thread_start_us":65,"threads_started":1}
I20260924 13:47:49.940599 21415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:49.940384 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:35859 (local address 127.13.149.194:43855)
0924 13:47:49.940519 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0924 13:47:49.940554 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.149.254:35859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":53,"threads_started":1}
I20260924 13:47:49.940893 21414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:49.940319 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:35859 (local address 127.13.149.195:36721)
0924 13:47:49.940816 (+ 497us) negotiation.cc:107] Waiting for socket to connect
0924 13:47:49.940852 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.149.254:35859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":480,"thread_start_us":62,"threads_started":1}
I20260924 13:47:50.941602 21416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:50.940859 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:35859 (local address 127.13.149.193:41691)
0924 13:47:50.941369 (+ 510us) negotiation.cc:107] Waiting for socket to connect
0924 13:47:50.941539 (+ 170us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.149.254:35859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":490,"thread_start_us":90,"threads_started":1}
I20260924 13:47:50.941654 21418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:50.941280 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:35859 (local address 127.13.149.195:35781)
0924 13:47:50.941568 (+ 288us) negotiation.cc:107] Waiting for socket to connect
0924 13:47:50.941607 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.149.254:35859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":269,"thread_start_us":61,"threads_started":1}
I20260924 13:47:50.941938 21417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:50.941156 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:35859 (local address 127.13.149.194:54775)
0924 13:47:50.941828 (+ 672us) negotiation.cc:107] Waiting for socket to connect
0924 13:47:50.941900 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.149.254:35859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":651,"thread_start_us":61,"threads_started":1}
I20260924 13:47:51.942278 21420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:51.942012 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:35859 (local address 127.13.149.195:59173)
0924 13:47:51.942197 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0924 13:47:51.942224 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.149.254:35859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":40,"threads_started":1}
I20260924 13:47:51.942278 21419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:51.941953 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:35859 (local address 127.13.149.193:42685)
0924 13:47:51.942115 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0924 13:47:51.942224 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.149.254:35859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":85,"threads_started":1}
I20260924 13:47:51.942473 21421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:51.942285 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:35859 (local address 127.13.149.194:34983)
0924 13:47:51.942390 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0924 13:47:51.942425 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.149.254:35859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":88,"thread_start_us":50,"threads_started":1}
I20260924 13:47:52.942973 21422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:52.942760 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:35859 (local address 127.13.149.193:48827)
0924 13:47:52.942875 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0924 13:47:52.942916 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.149.254:35859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96,"thread_start_us":66,"threads_started":1}
I20260924 13:47:52.943137 21423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:52.942842 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:35859 (local address 127.13.149.194:35897)
0924 13:47:52.943003 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0924 13:47:52.943055 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.149.254:35859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":52,"threads_started":1}
I20260924 13:47:52.943192 21424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:52.942949 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:35859 (local address 127.13.149.195:32857)
0924 13:47:52.943108 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0924 13:47:52.943141 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.149.254:35859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":41,"threads_started":1}
I20260924 13:47:53.720659 21425 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1d4faf110d7a41cc8a2113c5b74a298a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9976d5567e674e309eecdd83a7ced13a)
I20260924 13:47:53.720775 21425 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1d4faf110d7a41cc8a2113c5b74a298a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "9976d5567e674e309eecdd83a7ced13a" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 35859 } } peers { permanent_uuid: "1d4faf110d7a41cc8a2113c5b74a298a" member_type: VOTER last_known_addr { host: "127.13.149.253" port: 38973 } attrs { promote: false } } peers { permanent_uuid: "e42b5e67f6b6495080300913733a80db" member_type: NON_VOTER last_known_addr { host: "127.13.149.252" port: 35717 } attrs { promote: true } }
I20260924 13:47:53.720988 21425 leader_election.cc:288] T 00000000000000000000000000000000 P 1d4faf110d7a41cc8a2113c5b74a298a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9976d5567e674e309eecdd83a7ced13a (127.13.149.254:35859)
I20260924 13:47:53.721369 21426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:53.721148 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:35859 (local address 127.0.0.1:54378)
0924 13:47:53.721288 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0924 13:47:53.721333 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.149.254:35859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":60,"threads_started":1}
W20260924 13:47:53.721472 20658 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.13.149.254:35859: connect: Connection refused (error 111)
I20260924 13:47:53.721837 21426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:53.721745 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:35859 (local address 127.0.0.1:54392)
0924 13:47:53.721793 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0924 13:47:53.721818 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.149.254:35859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260924 13:47:53.721933 20658 leader_election.cc:334] T 00000000000000000000000000000000 P 1d4faf110d7a41cc8a2113c5b74a298a [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 9976d5567e674e309eecdd83a7ced13a (127.13.149.254:35859): Network error: Client connection negotiation failed: client connection to 127.13.149.254:35859: connect: Connection refused (error 111)
I20260924 13:47:53.721999 20658 leader_election.cc:302] T 00000000000000000000000000000000 P 1d4faf110d7a41cc8a2113c5b74a298a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 2 responses out of 2 voters: 1 yes votes; 1 no votes. yes voters: 1d4faf110d7a41cc8a2113c5b74a298a; no voters: 9976d5567e674e309eecdd83a7ced13a
I20260924 13:47:53.722084 21425 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 1d4faf110d7a41cc8a2113c5b74a298a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260924 13:47:53.943718 21428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:53.943495 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:35859 (local address 127.13.149.194:56289)
0924 13:47:53.943618 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0924 13:47:53.943661 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.149.254:35859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":62,"threads_started":1}
I20260924 13:47:53.943747 21429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:53.943503 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:35859 (local address 127.13.149.193:56645)
0924 13:47:53.943683 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0924 13:47:53.943712 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.149.254:35859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":68,"threads_started":1}
I20260924 13:47:53.943858 21430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:53.943586 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:35859 (local address 127.13.149.195:58701)
0924 13:47:53.943674 (+ 88us) negotiation.cc:107] Waiting for socket to connect
0924 13:47:53.943710 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.149.254:35859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71,"thread_start_us":75,"threads_started":1}
I20260924 13:47:54.944379 21432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:54.944088 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:35859 (local address 127.13.149.193:52727)
0924 13:47:54.944268 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0924 13:47:54.944314 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.149.254:35859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":109,"threads_started":1}
I20260924 13:47:54.944417 21431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:54.944130 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:35859 (local address 127.13.149.194:47635)
0924 13:47:54.944353 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0924 13:47:54.944382 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.149.254:35859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":60,"threads_started":1}
I20260924 13:47:54.944525 21433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:54.944326 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:35859 (local address 127.13.149.195:47531)
0924 13:47:54.944437 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0924 13:47:54.944473 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.149.254:35859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":94,"thread_start_us":45,"threads_started":1}
I20260924 13:47:55.945093 21434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:55.944782 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:35859 (local address 127.13.149.194:53493)
0924 13:47:55.944982 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0924 13:47:55.945030 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.149.254:35859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":67,"threads_started":1}
I20260924 13:47:55.945251 21435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:55.944894 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:35859 (local address 127.13.149.195:39437)
0924 13:47:55.945046 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0924 13:47:55.945089 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.149.254:35859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":90,"threads_started":1}
I20260924 13:47:55.945380 21436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:55.945162 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:35859 (local address 127.13.149.193:47211)
0924 13:47:55.945298 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0924 13:47:55.945332 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.149.254:35859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":61,"threads_started":1}
I20260924 13:47:56.945928 21439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:56.945748 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:35859 (local address 127.13.149.195:35345)
0924 13:47:56.945862 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0924 13:47:56.945893 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.149.254:35859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95,"thread_start_us":65,"threads_started":1}
I20260924 13:47:56.945924 21438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:56.945711 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:35859 (local address 127.13.149.193:46513)
0924 13:47:56.945837 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0924 13:47:56.945875 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.149.254:35859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":53,"threads_started":1}
I20260924 13:47:56.945906 21437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:56.945671 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:35859 (local address 127.13.149.194:35129)
0924 13:47:56.945798 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0924 13:47:56.945847 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.149.254:35859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":64,"threads_started":1}
I20260924 13:47:57.812258 21440 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1d4faf110d7a41cc8a2113c5b74a298a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9976d5567e674e309eecdd83a7ced13a)
I20260924 13:47:57.812366 21440 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1d4faf110d7a41cc8a2113c5b74a298a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "9976d5567e674e309eecdd83a7ced13a" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 35859 } } peers { permanent_uuid: "1d4faf110d7a41cc8a2113c5b74a298a" member_type: VOTER last_known_addr { host: "127.13.149.253" port: 38973 } attrs { promote: false } } peers { permanent_uuid: "e42b5e67f6b6495080300913733a80db" member_type: NON_VOTER last_known_addr { host: "127.13.149.252" port: 35717 } attrs { promote: true } }
I20260924 13:47:57.812532 21440 leader_election.cc:288] T 00000000000000000000000000000000 P 1d4faf110d7a41cc8a2113c5b74a298a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9976d5567e674e309eecdd83a7ced13a (127.13.149.254:35859)
I20260924 13:47:57.812860 21441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:57.812692 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:35859 (local address 127.0.0.1:54586)
0924 13:47:57.812787 (+ 95us) negotiation.cc:107] Waiting for socket to connect
0924 13:47:57.812822 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.149.254:35859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78,"thread_start_us":42,"threads_started":1}
I20260924 13:47:57.813246 21441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:57.813162 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:35859 (local address 127.0.0.1:54592)
0924 13:47:57.813203 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0924 13:47:57.813228 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.149.254:35859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260924 13:47:57.813314 20658 leader_election.cc:334] T 00000000000000000000000000000000 P 1d4faf110d7a41cc8a2113c5b74a298a [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 9976d5567e674e309eecdd83a7ced13a (127.13.149.254:35859): Network error: Client connection negotiation failed: client connection to 127.13.149.254:35859: connect: Connection refused (error 111)
I20260924 13:47:57.813356 20658 leader_election.cc:302] T 00000000000000000000000000000000 P 1d4faf110d7a41cc8a2113c5b74a298a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 2 responses out of 2 voters: 1 yes votes; 1 no votes. yes voters: 1d4faf110d7a41cc8a2113c5b74a298a; no voters: 9976d5567e674e309eecdd83a7ced13a
I20260924 13:47:57.813445 21440 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 1d4faf110d7a41cc8a2113c5b74a298a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260924 13:47:57.946614 21443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:57.946362 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:35859 (local address 127.13.149.193:52397)
0924 13:47:57.946503 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0924 13:47:57.946554 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.149.254:35859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":84,"threads_started":1}
I20260924 13:47:57.946679 21444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:57.946486 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:35859 (local address 127.13.149.195:44447)
0924 13:47:57.946611 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0924 13:47:57.946643 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.149.254:35859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":48,"threads_started":1}
I20260924 13:47:57.946902 21445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:57.946595 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:35859 (local address 127.13.149.194:41937)
0924 13:47:57.946820 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0924 13:47:57.946854 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.149.254:35859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":206,"thread_start_us":53,"threads_started":1}
I20260924 13:47:58.947425 21447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:58.947192 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:35859 (local address 127.13.149.193:33303)
0924 13:47:58.947335 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0924 13:47:58.947377 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.149.254:35859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":56,"threads_started":1}
I20260924 13:47:58.947413 21446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:58.947114 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:35859 (local address 127.13.149.195:40217)
0924 13:47:58.947255 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0924 13:47:58.947301 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.149.254:35859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":66,"threads_started":1}
I20260924 13:47:58.947669 21448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:58.947405 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:35859 (local address 127.13.149.194:44457)
0924 13:47:58.947557 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0924 13:47:58.947592 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.149.254:35859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":54,"threads_started":1}
I20260924 13:47:59.948012 21449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:59.947766 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:35859 (local address 127.13.149.195:35271)
0924 13:47:59.947901 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0924 13:47:59.947953 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.149.254:35859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":77,"threads_started":1}
I20260924 13:47:59.948220 21451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:59.948006 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:35859 (local address 127.13.149.194:56051)
0924 13:47:59.948139 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0924 13:47:59.948173 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.149.254:35859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":55,"threads_started":1}
I20260924 13:47:59.948256 21450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:47:59.947919 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:35859 (local address 127.13.149.193:37135)
0924 13:47:59.948191 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0924 13:47:59.948220 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.149.254:35859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":257,"thread_start_us":48,"threads_started":1}
I20260924 13:48:00.948894 21452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:48:00.948642 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:35859 (local address 127.13.149.194:56105)
0924 13:48:00.948786 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0924 13:48:00.948833 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.149.254:35859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":70,"threads_started":1}
I20260924 13:48:00.949254 21453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:48:00.948644 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:35859 (local address 127.13.149.193:47765)
0924 13:48:00.949165 (+ 521us) negotiation.cc:107] Waiting for socket to connect
0924 13:48:00.949207 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.149.254:35859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":501,"thread_start_us":75,"threads_started":1}
I20260924 13:48:00.949417 21454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:48:00.949063 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:35859 (local address 127.13.149.195:50803)
0924 13:48:00.949312 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0924 13:48:00.949360 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.149.254:35859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":227,"thread_start_us":55,"threads_started":1}
I20260924 13:48:01.949867 21455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:48:01.949493 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:35859 (local address 127.13.149.194:52379)
0924 13:48:01.949680 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0924 13:48:01.949804 (+ 124us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.149.254:35859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":65,"threads_started":1}
I20260924 13:48:01.949909 21456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:48:01.949732 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:35859 (local address 127.13.149.193:46773)
0924 13:48:01.949838 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0924 13:48:01.949869 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.149.254:35859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":91,"thread_start_us":46,"threads_started":1}
I20260924 13:48:01.953682 21457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:48:01.953400 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:35859 (local address 127.13.149.195:60247)
0924 13:48:01.953591 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0924 13:48:01.953635 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.149.254:35859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":73,"threads_started":1}
I20260924 13:48:02.950594 21458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:48:02.950250 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:35859 (local address 127.13.149.193:37073)
0924 13:48:02.950474 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0924 13:48:02.950533 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.149.254:35859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":78,"threads_started":1}
I20260924 13:48:02.950606 21459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:48:02.950414 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:35859 (local address 127.13.149.194:36049)
0924 13:48:02.950520 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0924 13:48:02.950557 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.149.254:35859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":87,"thread_start_us":42,"threads_started":1}
I20260924 13:48:02.954600 21460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:48:02.954260 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:35859 (local address 127.13.149.195:56873)
0924 13:48:02.954458 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0924 13:48:02.954533 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.149.254:35859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":67,"threads_started":1}
I20260924 13:48:03.951248 21461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:48:03.951018 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:35859 (local address 127.13.149.193:43803)
0924 13:48:03.951148 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0924 13:48:03.951196 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.149.254:35859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":65,"threads_started":1}
I20260924 13:48:03.951330 21462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:48:03.951066 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:35859 (local address 127.13.149.194:41751)
0924 13:48:03.951217 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0924 13:48:03.951277 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.149.254:35859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96,"thread_start_us":53,"threads_started":1}
I20260924 13:48:03.955487 21463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:48:03.955220 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:35859 (local address 127.13.149.195:57561)
0924 13:48:03.955384 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0924 13:48:03.955432 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.149.254:35859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":80,"threads_started":1}
I20260924 13:48:04.951962 21464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:48:04.951751 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:35859 (local address 127.13.149.194:59617)
0924 13:48:04.951879 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0924 13:48:04.951917 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.149.254:35859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":67,"threads_started":1}
I20260924 13:48:04.951972 21465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:48:04.951748 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:35859 (local address 127.13.149.193:36315)
0924 13:48:04.951881 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0924 13:48:04.951933 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.149.254:35859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":73,"threads_started":1}
I20260924 13:48:04.956212 21466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:48:04.955990 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:35859 (local address 127.13.149.195:44349)
0924 13:48:04.956124 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0924 13:48:04.956166 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.149.254:35859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":71,"threads_started":1}
I20260924 13:48:05.952628 21467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:48:05.952395 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:35859 (local address 127.13.149.194:45177)
0924 13:48:05.952536 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0924 13:48:05.952577 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.149.254:35859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":70,"threads_started":1}
I20260924 13:48:05.952775 21468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:48:05.952537 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:35859 (local address 127.13.149.193:50019)
0924 13:48:05.952631 (+ 94us) negotiation.cc:107] Waiting for socket to connect
0924 13:48:05.952664 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.149.254:35859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76,"thread_start_us":51,"threads_started":1}
I20260924 13:48:05.956954 21469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:48:05.956694 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:35859 (local address 127.13.149.195:38477)
0924 13:48:05.956825 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0924 13:48:05.956868 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.149.254:35859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":67,"threads_started":1}
I20260924 13:48:06.953363 21470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:48:06.953146 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:35859 (local address 127.13.149.194:59131)
0924 13:48:06.953279 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0924 13:48:06.953320 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.149.254:35859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":60,"threads_started":1}
I20260924 13:48:06.953374 21471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:48:06.953204 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:35859 (local address 127.13.149.193:48629)
0924 13:48:06.953303 (+ 99us) negotiation.cc:107] Waiting for socket to connect
0924 13:48:06.953336 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.149.254:35859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":84,"thread_start_us":48,"threads_started":1}
I20260924 13:48:06.957612 21472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:48:06.957352 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:35859 (local address 127.13.149.195:41281)
0924 13:48:06.957508 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0924 13:48:06.957557 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.149.254:35859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":71,"threads_started":1}
I20260924 13:48:07.954042 21473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:48:07.953813 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:35859 (local address 127.13.149.193:46251)
0924 13:48:07.953926 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0924 13:48:07.953995 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.149.254:35859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93,"thread_start_us":59,"threads_started":1}
I20260924 13:48:07.954093 21474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:48:07.953866 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:35859 (local address 127.13.149.194:49017)
0924 13:48:07.953982 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0924 13:48:07.954018 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.149.254:35859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":50,"threads_started":1}
I20260924 13:48:07.958266 21475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:48:07.958033 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:35859 (local address 127.13.149.195:50663)
0924 13:48:07.958152 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0924 13:48:07.958189 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.149.254:35859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":72,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:903: Failure
Failed
Bad status: Timed out: Could not connect to the cluster: ConnectToClusterRpc(addrs: 127.13.149.254:35859,127.13.149.253:38973,127.13.149.252:35717, num_attempts: 243) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.13.149.254:35859,127.13.149.253:38973,127.13.149.252:35717, num_attempts: 1)
I20260924 13:48:08.136026 13911 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJN9WRF/build/release/bin/kudu with pid 20818
I20260924 13:48:08.140642 13911 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJN9WRF/build/release/bin/kudu with pid 21046
I20260924 13:48:08.145008 13911 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJN9WRF/build/release/bin/kudu with pid 21182
I20260924 13:48:08.149788 13911 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJN9WRF/build/release/bin/kudu with pid 20538
I20260924 13:48:08.153703 13911 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJN9WRF/build/release/bin/kudu with pid 20725
2026-09-24T13:48:08Z chronyd exiting
I20260924 13:48:08.169178 13911 test_util.cc:175] -----------------------------------------------
I20260924 13:48:08.169242 13911 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790257629041645-13911-0
[ FAILED ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (33079 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790257629041645-13911-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 13:48:08 dist-test-slave-sjl2 krb5kdc[21478](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 13:48:08 dist-test-slave-sjl2 krb5kdc[21478](info): set up 2 sockets
Sep 24 13:48:08 dist-test-slave-sjl2 krb5kdc[21478](info): commencing operation
krb5kdc: starting...
W20260924 13:48:09.119037 13911 mini_kdc.cc:121] Time spent starting KDC: real 0.949s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790257629041645-13911-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790257629041645-13911-0/krb5kdc/test-user.keytab.
Sep 24 13:48:09 dist-test-slave-sjl2 krb5kdc[21478](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257689, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T13:48:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T13:48:09Z Disabled control of system clock
WARNING: no policy specified for oryx/127.13.149.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.13.149.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.13.149.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790257629041645-13911-0/krb5kdc/oryx_127.13.149.254.keytab.
Entry for principal oryx/127.13.149.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790257629041645-13911-0/krb5kdc/oryx_127.13.149.254.keytab.
I20260924 13:48:09.249671 13911 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJN9WRF/build/release/bin/kudu
/tmp/dist-test-taskJN9WRF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790257629041645-13911-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790257629041645-13911-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790257629041645-13911-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790257629041645-13911-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.149.254:42023
--webserver_interface=127.13.149.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790257629041645-13911-0/krb5kdc/oryx_127.13.149.254.keytab
--principal=oryx/127.13.149.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.149.212:44865
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.149.254:42023
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790257629041645-13911-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790257629041645-13911-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790257629041645-13911-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 13:48:09.334517 21491 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 13:48:09.334743 21491 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 13:48:09.334767 21491 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 13:48:09.335980 21491 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 13:48:09.336023 21491 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260924 13:48:09.336035 21491 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 13:48:09.336047 21491 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 13:48:09.336058 21491 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 13:48:09.336069 21491 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 13:48:09.337695 21491 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.149.212:44865
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790257629041645-13911-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790257629041645-13911-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.149.254:42023
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790257629041645-13911-0/krb5kdc/oryx_127.13.149.254.keytab
--principal=oryx/127.13.149.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.149.254:42023
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790257629041645-13911-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.149.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21491
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790257629041645-13911-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 12:29:37 UTC on ccycloud.quasar-iqzbxn.root.comops.site
build id 4670106
I20260924 13:48:09.337931 21491 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 13:48:09.338122 21491 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 13:48:09.343323 21496 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 13:48:09.343402 21497 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 13:48:09.343432 21491 server_base.cc:1034] running on GCE node
W20260924 13:48:09.343323 21499 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 13:48:09.343797 21491 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 13:48:09.344110 21491 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 13:48:09.345248 21491 hybrid_clock.cc:648] HybridClock initialized: now 1790257689345230 us; error 37 us; skew 500 ppm
Sep 24 13:48:09 dist-test-slave-sjl2 krb5kdc[21478](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257689, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.149.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 13:48:09.347829 21491 init.cc:377] Logged in from keytab as oryx/127.13.149.254@KRBTEST.COM (short username oryx)
I20260924 13:48:09.348351 21491 webserver.cc:460] Webserver started at http://127.13.149.254:34223/ using document root <none> and password file <none>
I20260924 13:48:09.348596 21491 fs_manager.cc:359] Metadata directory not provided
I20260924 13:48:09.348649 21491 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 13:48:09.348770 21491 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 13:48:09.349989 21491 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790257629041645-13911-0/minicluster-data/master-0/data/instance:
uuid: "9f6c1d2f544b490eb7522be8ff50ceba"
format_stamp: "Formatted at 2026-09-24 13:48:09 on dist-test-slave-sjl2"
server_key: "66ddc6856076b536da3f80118d162d6d"
server_key_iv: "87ea1744d8a40cd182c1384246f6c5c9"
server_key_version: "encryptionkey@0"
I20260924 13:48:09.350358 21491 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790257629041645-13911-0/minicluster-data/master-0/wal/instance:
uuid: "9f6c1d2f544b490eb7522be8ff50ceba"
format_stamp: "Formatted at 2026-09-24 13:48:09 on dist-test-slave-sjl2"
server_key: "66ddc6856076b536da3f80118d162d6d"
server_key_iv: "87ea1744d8a40cd182c1384246f6c5c9"
server_key_version: "encryptionkey@0"
I20260924 13:48:09.351881 21491 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260924 13:48:09.352715 21506 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 13:48:09.352978 21491 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260924 13:48:09.353080 21491 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790257629041645-13911-0/minicluster-data/master-0/data,/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790257629041645-13911-0/minicluster-data/master-0/wal
uuid: "9f6c1d2f544b490eb7522be8ff50ceba"
format_stamp: "Formatted at 2026-09-24 13:48:09 on dist-test-slave-sjl2"
server_key: "66ddc6856076b536da3f80118d162d6d"
server_key_iv: "87ea1744d8a40cd182c1384246f6c5c9"
server_key_version: "encryptionkey@0"
I20260924 13:48:09.353148 21491 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790257629041645-13911-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790257629041645-13911-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790257629041645-13911-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 13:48:09.368420 21491 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 13:48:09.370445 21491 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 13:48:09.370602 21491 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 13:48:09.375020 21491 rpc_server.cc:307] RPC server started. Bound to: 127.13.149.254:42023
I20260924 13:48:09.375098 21568 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.149.254:42023 every 8 connection(s)
I20260924 13:48:09.375385 21491 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790257629041645-13911-0/minicluster-data/master-0/data/info.pb
I20260924 13:48:09.376101 21569 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 13:48:09.378598 21569 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9f6c1d2f544b490eb7522be8ff50ceba: Bootstrap starting.
I20260924 13:48:09.379225 21569 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9f6c1d2f544b490eb7522be8ff50ceba: Neither blocks nor log segments found. Creating new log.
I20260924 13:48:09.379520 21569 log.cc:824] T 00000000000000000000000000000000 P 9f6c1d2f544b490eb7522be8ff50ceba: Log is configured to *not* fsync() on all Append() calls
I20260924 13:48:09.380321 21569 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9f6c1d2f544b490eb7522be8ff50ceba: No bootstrap required, opened a new log
I20260924 13:48:09.382567 21569 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9f6c1d2f544b490eb7522be8ff50ceba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f6c1d2f544b490eb7522be8ff50ceba" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 42023 } }
I20260924 13:48:09.382755 21569 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9f6c1d2f544b490eb7522be8ff50ceba [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 13:48:09.382781 21569 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9f6c1d2f544b490eb7522be8ff50ceba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9f6c1d2f544b490eb7522be8ff50ceba, State: Initialized, Role: FOLLOWER
I20260924 13:48:09.382927 21569 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9f6c1d2f544b490eb7522be8ff50ceba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f6c1d2f544b490eb7522be8ff50ceba" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 42023 } }
I20260924 13:48:09.383015 21569 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9f6c1d2f544b490eb7522be8ff50ceba [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 13:48:09.383044 21569 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9f6c1d2f544b490eb7522be8ff50ceba [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 13:48:09.383081 21569 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9f6c1d2f544b490eb7522be8ff50ceba [term 0 FOLLOWER]: Advancing to term 1
I20260924 13:48:09.383739 21569 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9f6c1d2f544b490eb7522be8ff50ceba [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f6c1d2f544b490eb7522be8ff50ceba" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 42023 } }
I20260924 13:48:09.383843 21569 leader_election.cc:302] T 00000000000000000000000000000000 P 9f6c1d2f544b490eb7522be8ff50ceba [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9f6c1d2f544b490eb7522be8ff50ceba; no voters:
I20260924 13:48:09.384040 21569 leader_election.cc:288] T 00000000000000000000000000000000 P 9f6c1d2f544b490eb7522be8ff50ceba [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 13:48:09.384131 21572 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9f6c1d2f544b490eb7522be8ff50ceba [term 1 FOLLOWER]: Leader election won for term 1
I20260924 13:48:09.384291 21572 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9f6c1d2f544b490eb7522be8ff50ceba [term 1 LEADER]: Becoming Leader. State: Replica: 9f6c1d2f544b490eb7522be8ff50ceba, State: Running, Role: LEADER
I20260924 13:48:09.384392 21569 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9f6c1d2f544b490eb7522be8ff50ceba [sys.catalog]: configured and running, proceeding with master startup.
I20260924 13:48:09.384420 21572 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9f6c1d2f544b490eb7522be8ff50ceba [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f6c1d2f544b490eb7522be8ff50ceba" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 42023 } }
I20260924 13:48:09.384806 21573 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9f6c1d2f544b490eb7522be8ff50ceba [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9f6c1d2f544b490eb7522be8ff50ceba" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f6c1d2f544b490eb7522be8ff50ceba" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 42023 } } }
I20260924 13:48:09.384989 21573 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9f6c1d2f544b490eb7522be8ff50ceba [sys.catalog]: This master's current role is: LEADER
I20260924 13:48:09.385390 13911 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJN9WRF/build/release/bin/kudu as pid 21491
I20260924 13:48:09.385502 13911 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790257629041645-13911-0/minicluster-data/master-0/wal/instance
I20260924 13:48:09.385591 21581 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 13:48:09.385684 13911 external_mini_cluster.cc:1428] Setting key 4cf7ecaf4a5c9f1cf015aa3ba73c0747
I20260924 13:48:09.384814 21574 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9f6c1d2f544b490eb7522be8ff50ceba [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9f6c1d2f544b490eb7522be8ff50ceba. Latest consensus state: current_term: 1 leader_uuid: "9f6c1d2f544b490eb7522be8ff50ceba" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f6c1d2f544b490eb7522be8ff50ceba" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 42023 } } }
I20260924 13:48:09.386365 21574 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9f6c1d2f544b490eb7522be8ff50ceba [sys.catalog]: This master's current role is: LEADER
I20260924 13:48:09.386632 21581 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 13:48:09.388216 21581 catalog_manager.cc:1347] Generated new cluster ID: 5d9fd4003e46492292c0b9a8c645483b
I20260924 13:48:09.388273 21581 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 24 13:48:09 dist-test-slave-sjl2 krb5kdc[21478](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257689, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.13.149.254@KRBTEST.COM
I20260924 13:48:09.397172 21590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:48:09.386228 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46460 (local address 127.13.149.254:42023)
0924 13:48:09.386362 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0924 13:48:09.386373 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0924 13:48:09.388014 (+ 1641us) server_negotiation.cc:407] Connection header received
0924 13:48:09.388146 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:48:09.388154 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:48:09.388212 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:48:09.388357 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:48:09.389050 (+ 693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:48:09.389990 (+ 940us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:48:09.390728 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:48:09.391193 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:48:09.391303 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:48:09.393818 (+ 2515us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:48:09.393824 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:48:09.393824 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 13:48:09.393837 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:48:09.395534 (+ 1697us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:48:09.395925 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:48:09.395929 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:48:09.395930 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:48:09.396048 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:48:09.396232 (+ 184us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:48:09.396233 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:48:09.396233 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:48:09.396519 (+ 286us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:48:09.396670 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0924 13:48:09.396777 (+ 107us) server_negotiation.cc:299] Negotiation successful
0924 13:48:09.396952 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84,"thread_start_us":57,"threads_started":1}
I20260924 13:48:09.427536 21581 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 13:48:09.428105 21581 catalog_manager.cc:1504] Loading token signing keys...
I20260924 13:48:09.432466 21581 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9f6c1d2f544b490eb7522be8ff50ceba: Generated new TSK 0
I20260924 13:48:09.432673 21581 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.13.149.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.13.149.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.13.149.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790257629041645-13911-0/krb5kdc/oryx_127.13.149.193.keytab.
Entry for principal oryx/127.13.149.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790257629041645-13911-0/krb5kdc/oryx_127.13.149.193.keytab.
I20260924 13:48:09.464607 13911 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJN9WRF/build/release/bin/kudu
/tmp/dist-test-taskJN9WRF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790257629041645-13911-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790257629041645-13911-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790257629041645-13911-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790257629041645-13911-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.149.193:0
--local_ip_for_outbound_sockets=127.13.149.193
--webserver_interface=127.13.149.193
--webserver_port=0
--tserver_master_addrs=127.13.149.254:42023
--keytab_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790257629041645-13911-0/krb5kdc/oryx_127.13.149.193.keytab
--principal=oryx/127.13.149.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.149.212:44865
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790257629041645-13911-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790257629041645-13911-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790257629041645-13911-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 13:48:09.588141 21595 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 13:48:09.588490 21595 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 13:48:09.588536 21595 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 13:48:09.590344 21595 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 13:48:09.590418 21595 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 13:48:09.590451 21595 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260924 13:48:09.590477 21595 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.149.193
I20260924 13:48:09.592257 21595 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.149.212:44865
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790257629041645-13911-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790257629041645-13911-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790257629041645-13911-0/krb5kdc/oryx_127.13.149.193.keytab
--principal=oryx/127.13.149.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.149.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790257629041645-13911-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.149.193
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.13.149.254:42023
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21595
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.149.193
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790257629041645-13911-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 12:29:37 UTC on ccycloud.quasar-iqzbxn.root.comops.site
build id 4670106
I20260924 13:48:09.592527 21595 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 13:48:09.592762 21595 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 13:48:09.597743 21600 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 13:48:09.597939 21603 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 13:48:09.597726 21601 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 13:48:09.598675 21595 server_base.cc:1034] running on GCE node
I20260924 13:48:09.598922 21595 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 13:48:09.599176 21595 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 13:48:09.602445 21595 hybrid_clock.cc:648] HybridClock initialized: now 1790257689602419 us; error 51 us; skew 500 ppm
Sep 24 13:48:09 dist-test-slave-sjl2 krb5kdc[21478](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257689, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.149.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 13:48:09.606046 21595 init.cc:377] Logged in from keytab as oryx/127.13.149.193@KRBTEST.COM (short username oryx)
I20260924 13:48:09.606535 21595 webserver.cc:460] Webserver started at http://127.13.149.193:34931/ using document root <none> and password file <none>
I20260924 13:48:09.606804 21595 fs_manager.cc:359] Metadata directory not provided
I20260924 13:48:09.606873 21595 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 13:48:09.607040 21595 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 13:48:09.608189 21595 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790257629041645-13911-0/minicluster-data/ts-0/data/instance:
uuid: "a66912dfe2404677a7a20ec0462d8ff0"
format_stamp: "Formatted at 2026-09-24 13:48:09 on dist-test-slave-sjl2"
server_key: "6b9fbab829bdd20ac795921dd15cb885"
server_key_iv: "025d18ed2a38fa7cf733d5622aa8e8b2"
server_key_version: "encryptionkey@0"
I20260924 13:48:09.608544 21595 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790257629041645-13911-0/minicluster-data/ts-0/wal/instance:
uuid: "a66912dfe2404677a7a20ec0462d8ff0"
format_stamp: "Formatted at 2026-09-24 13:48:09 on dist-test-slave-sjl2"
server_key: "6b9fbab829bdd20ac795921dd15cb885"
server_key_iv: "025d18ed2a38fa7cf733d5622aa8e8b2"
server_key_version: "encryptionkey@0"
I20260924 13:48:09.610117 21595 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260924 13:48:09.611258 21610 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 13:48:09.611435 21595 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260924 13:48:09.611542 21595 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790257629041645-13911-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790257629041645-13911-0/minicluster-data/ts-0/wal
uuid: "a66912dfe2404677a7a20ec0462d8ff0"
format_stamp: "Formatted at 2026-09-24 13:48:09 on dist-test-slave-sjl2"
server_key: "6b9fbab829bdd20ac795921dd15cb885"
server_key_iv: "025d18ed2a38fa7cf733d5622aa8e8b2"
server_key_version: "encryptionkey@0"
I20260924 13:48:09.611629 21595 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790257629041645-13911-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790257629041645-13911-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790257629041645-13911-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 13:48:09.643697 21595 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 13:48:09.646184 21595 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 13:48:09.646315 21595 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 13:48:09.646898 21595 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 13:48:09.646940 21595 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 13:48:09.646979 21595 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 13:48:09.647007 21595 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 24 13:48:09 dist-test-slave-sjl2 krb5kdc[21478](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257689, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.149.193@KRBTEST.COM for oryx/127.13.149.254@KRBTEST.COM
I20260924 13:48:09.654978 21595 rpc_server.cc:307] RPC server started. Bound to: 127.13.149.193:45785
I20260924 13:48:09.655114 21725 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.149.193:45785 every 8 connection(s)
I20260924 13:48:09.655478 21595 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790257629041645-13911-0/minicluster-data/ts-0/data/info.pb
I20260924 13:48:09.658913 21590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:48:09.647568 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.149.193:42307 (local address 127.13.149.254:42023)
0924 13:48:09.647631 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0924 13:48:09.647634 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 13:48:09.647813 (+ 179us) server_negotiation.cc:407] Connection header received
0924 13:48:09.648458 (+ 645us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:48:09.648459 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:48:09.648473 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:48:09.648528 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:48:09.649586 (+ 1058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:48:09.650469 (+ 883us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:48:09.651655 (+ 1186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:48:09.651901 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:48:09.651964 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:48:09.654941 (+ 2977us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:48:09.654944 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:48:09.654945 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 13:48:09.654959 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:48:09.657173 (+ 2214us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:48:09.657731 (+ 558us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:48:09.657733 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:48:09.657733 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:48:09.657809 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:48:09.658092 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:48:09.658092 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:48:09.658093 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:48:09.658239 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:48:09.658333 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0924 13:48:09.658669 (+ 336us) server_negotiation.cc:299] Negotiation successful
0924 13:48:09.658801 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260924 13:48:09.659113 21619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:48:09.647486 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:42023 (local address 127.13.149.193:42307)
0924 13:48:09.647689 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0924 13:48:09.647729 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0924 13:48:09.648347 (+ 618us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:48:09.648634 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:48:09.648640 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:48:09.649081 (+ 441us) client_negotiation.cc:189] Negotiated authn=SASL
0924 13:48:09.649410 (+ 329us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:48:09.649418 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:48:09.650601 (+ 1183us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:48:09.650602 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:48:09.651562 (+ 960us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:48:09.651566 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:48:09.652012 (+ 446us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:48:09.652013 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:48:09.652163 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:48:09.652736 (+ 573us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 13:48:09.652746 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0924 13:48:09.654642 (+ 1896us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 13:48:09.657308 (+ 2666us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 13:48:09.657320 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 13:48:09.657323 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0924 13:48:09.657636 (+ 313us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 13:48:09.657876 (+ 240us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 13:48:09.657877 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 13:48:09.657878 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 13:48:09.658024 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 13:48:09.658330 (+ 306us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 13:48:09.658338 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 13:48:09.658518 (+ 180us) client_negotiation.cc:769] Sending connection context
0924 13:48:09.658628 (+ 110us) client_negotiation.cc:240] Negotiation successful
0924 13:48:09.658826 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":79,"threads_started":1}
I20260924 13:48:09.659747 21726 heartbeater.cc:347] Connected to a master server at 127.13.149.254:42023
I20260924 13:48:09.659852 21726 heartbeater.cc:464] Registering TS with master...
I20260924 13:48:09.660064 21726 heartbeater.cc:511] Master 127.13.149.254:42023 requested a full tablet report, sending...
I20260924 13:48:09.661243 21522 ts_manager.cc:194] Registered new tserver with Master: a66912dfe2404677a7a20ec0462d8ff0 (127.13.149.193:45785)
I20260924 13:48:09.661445 13911 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJN9WRF/build/release/bin/kudu as pid 21595
I20260924 13:48:09.661552 13911 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790257629041645-13911-0/minicluster-data/ts-0/wal/instance
I20260924 13:48:09.661760 13911 external_mini_cluster.cc:1428] Setting key 41b590920397f820edbfb837fb7692af
I20260924 13:48:09.662259 21522 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.13.149.193@KRBTEST.COM'} at 127.13.149.193:42307
I20260924 13:48:09.668093 21590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:48:09.661130 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.149.193:55739 (local address 127.13.149.254:42023)
0924 13:48:09.661183 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0924 13:48:09.661185 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:48:09.661306 (+ 121us) server_negotiation.cc:407] Connection header received
0924 13:48:09.661374 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:48:09.661375 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:48:09.661387 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:48:09.661445 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:48:09.661832 (+ 387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:48:09.662630 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:48:09.663704 (+ 1074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:48:09.664018 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:48:09.664084 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:48:09.665323 (+ 1239us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:48:09.665325 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:48:09.665326 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 13:48:09.665334 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:48:09.666917 (+ 1583us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:48:09.667318 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:48:09.667319 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:48:09.667319 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:48:09.667376 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:48:09.667605 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:48:09.667605 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:48:09.667605 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:48:09.667721 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:48:09.667771 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0924 13:48:09.667945 (+ 174us) server_negotiation.cc:299] Negotiation successful
0924 13:48:09.668017 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260924 13:48:09.668159 21732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:48:09.661053 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:42023 (local address 127.13.149.193:55739)
0924 13:48:09.661226 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0924 13:48:09.661238 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0924 13:48:09.661308 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:48:09.661452 (+ 144us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:48:09.661453 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:48:09.661677 (+ 224us) client_negotiation.cc:189] Negotiated authn=SASL
0924 13:48:09.661747 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:48:09.661749 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:48:09.662730 (+ 981us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:48:09.662732 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:48:09.663596 (+ 864us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:48:09.663600 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:48:09.664104 (+ 504us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:48:09.664105 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:48:09.664179 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:48:09.664741 (+ 562us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 13:48:09.664748 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0924 13:48:09.665228 (+ 480us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 13:48:09.667013 (+ 1785us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 13:48:09.667014 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 13:48:09.667014 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 13:48:09.667222 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 13:48:09.667442 (+ 220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 13:48:09.667443 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 13:48:09.667443 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 13:48:09.667518 (+ 75us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 13:48:09.667779 (+ 261us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 13:48:09.667780 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 13:48:09.667843 (+ 63us) client_negotiation.cc:769] Sending connection context
0924 13:48:09.667923 (+ 80us) client_negotiation.cc:240] Negotiation successful
0924 13:48:09.668048 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":99,"threads_started":1}
WARNING: no policy specified for oryx/127.13.149.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.13.149.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.13.149.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790257629041645-13911-0/krb5kdc/oryx_127.13.149.194.keytab.
Entry for principal oryx/127.13.149.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790257629041645-13911-0/krb5kdc/oryx_127.13.149.194.keytab.
I20260924 13:48:09.692888 13911 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJN9WRF/build/release/bin/kudu
/tmp/dist-test-taskJN9WRF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790257629041645-13911-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790257629041645-13911-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790257629041645-13911-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790257629041645-13911-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.149.194:0
--local_ip_for_outbound_sockets=127.13.149.194
--webserver_interface=127.13.149.194
--webserver_port=0
--tserver_master_addrs=127.13.149.254:42023
--keytab_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790257629041645-13911-0/krb5kdc/oryx_127.13.149.194.keytab
--principal=oryx/127.13.149.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.149.212:44865
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790257629041645-13911-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790257629041645-13911-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790257629041645-13911-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 13:48:09.778857 21735 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 13:48:09.779126 21735 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 13:48:09.779156 21735 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 13:48:09.780510 21735 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 13:48:09.780561 21735 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 13:48:09.780584 21735 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260924 13:48:09.780603 21735 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.149.194
I20260924 13:48:09.782418 21735 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.149.212:44865
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790257629041645-13911-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790257629041645-13911-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790257629041645-13911-0/krb5kdc/oryx_127.13.149.194.keytab
--principal=oryx/127.13.149.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.149.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790257629041645-13911-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.149.194
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.13.149.254:42023
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21735
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.149.194
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790257629041645-13911-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 12:29:37 UTC on ccycloud.quasar-iqzbxn.root.comops.site
build id 4670106
I20260924 13:48:09.782662 21735 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 13:48:09.782874 21735 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 13:48:09.788522 21743 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 13:48:09.788542 21741 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 13:48:09.788519 21740 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 13:48:09.788754 21735 server_base.cc:1034] running on GCE node
I20260924 13:48:09.789105 21735 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 13:48:09.789368 21735 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 13:48:09.790524 21735 hybrid_clock.cc:648] HybridClock initialized: now 1790257689790519 us; error 55 us; skew 500 ppm
Sep 24 13:48:09 dist-test-slave-sjl2 krb5kdc[21478](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257689, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.149.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 13:48:09.793334 21735 init.cc:377] Logged in from keytab as oryx/127.13.149.194@KRBTEST.COM (short username oryx)
I20260924 13:48:09.793800 21735 webserver.cc:460] Webserver started at http://127.13.149.194:45899/ using document root <none> and password file <none>
I20260924 13:48:09.794015 21735 fs_manager.cc:359] Metadata directory not provided
I20260924 13:48:09.794067 21735 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 13:48:09.794221 21735 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 13:48:09.795324 21735 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790257629041645-13911-0/minicluster-data/ts-1/data/instance:
uuid: "74fb0a58bbc04969a2fbe9c044f837ae"
format_stamp: "Formatted at 2026-09-24 13:48:09 on dist-test-slave-sjl2"
server_key: "20c0bb9bb27b07abf186e1d20f78cd85"
server_key_iv: "a299ec3e2027c5346268473216438711"
server_key_version: "encryptionkey@0"
I20260924 13:48:09.795681 21735 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790257629041645-13911-0/minicluster-data/ts-1/wal/instance:
uuid: "74fb0a58bbc04969a2fbe9c044f837ae"
format_stamp: "Formatted at 2026-09-24 13:48:09 on dist-test-slave-sjl2"
server_key: "20c0bb9bb27b07abf186e1d20f78cd85"
server_key_iv: "a299ec3e2027c5346268473216438711"
server_key_version: "encryptionkey@0"
I20260924 13:48:09.797284 21735 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260924 13:48:09.798095 21750 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 13:48:09.798249 21735 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260924 13:48:09.798339 21735 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790257629041645-13911-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790257629041645-13911-0/minicluster-data/ts-1/wal
uuid: "74fb0a58bbc04969a2fbe9c044f837ae"
format_stamp: "Formatted at 2026-09-24 13:48:09 on dist-test-slave-sjl2"
server_key: "20c0bb9bb27b07abf186e1d20f78cd85"
server_key_iv: "a299ec3e2027c5346268473216438711"
server_key_version: "encryptionkey@0"
I20260924 13:48:09.798409 21735 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790257629041645-13911-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790257629041645-13911-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790257629041645-13911-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 13:48:09.810351 21735 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 13:48:09.812579 21735 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 13:48:09.812757 21735 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 13:48:09.813313 21735 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 13:48:09.813361 21735 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 13:48:09.813390 21735 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 13:48:09.813407 21735 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 24 13:48:09 dist-test-slave-sjl2 krb5kdc[21478](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257689, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.149.194@KRBTEST.COM for oryx/127.13.149.254@KRBTEST.COM
I20260924 13:48:09.820465 21735 rpc_server.cc:307] RPC server started. Bound to: 127.13.149.194:46159
I20260924 13:48:09.820549 21865 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.149.194:46159 every 8 connection(s)
I20260924 13:48:09.820896 21735 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790257629041645-13911-0/minicluster-data/ts-1/data/info.pb
I20260924 13:48:09.823156 21590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:48:09.813813 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.149.194:47527 (local address 127.13.149.254:42023)
0924 13:48:09.813878 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0924 13:48:09.813880 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:48:09.814057 (+ 177us) server_negotiation.cc:407] Connection header received
0924 13:48:09.814748 (+ 691us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:48:09.814750 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:48:09.814764 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:48:09.814850 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:48:09.815710 (+ 860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:48:09.816489 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:48:09.817732 (+ 1243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:48:09.818033 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:48:09.818122 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:48:09.820558 (+ 2436us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:48:09.820560 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:48:09.820560 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 13:48:09.820570 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:48:09.821984 (+ 1414us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:48:09.822384 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:48:09.822385 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:48:09.822385 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:48:09.822431 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:48:09.822622 (+ 191us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:48:09.822622 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:48:09.822622 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:48:09.822730 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:48:09.822775 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0924 13:48:09.823009 (+ 234us) server_negotiation.cc:299] Negotiation successful
0924 13:48:09.823077 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260924 13:48:09.823359 21759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:48:09.813745 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:42023 (local address 127.13.149.194:47527)
0924 13:48:09.813910 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0924 13:48:09.813953 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0924 13:48:09.814635 (+ 682us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:48:09.814914 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:48:09.814919 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:48:09.815285 (+ 366us) client_negotiation.cc:189] Negotiated authn=SASL
0924 13:48:09.815592 (+ 307us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:48:09.815599 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:48:09.816600 (+ 1001us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:48:09.816602 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:48:09.817639 (+ 1037us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:48:09.817643 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:48:09.818146 (+ 503us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:48:09.818147 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:48:09.818333 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:48:09.818974 (+ 641us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 13:48:09.818984 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0924 13:48:09.820449 (+ 1465us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 13:48:09.822079 (+ 1630us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 13:48:09.822086 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 13:48:09.822088 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 13:48:09.822314 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 13:48:09.822476 (+ 162us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 13:48:09.822476 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 13:48:09.822477 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 13:48:09.822574 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 13:48:09.822772 (+ 198us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 13:48:09.822775 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 13:48:09.822908 (+ 133us) client_negotiation.cc:769] Sending connection context
0924 13:48:09.822990 (+ 82us) client_negotiation.cc:240] Negotiation successful
0924 13:48:09.823137 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":75,"threads_started":1}
I20260924 13:48:09.823791 21866 heartbeater.cc:347] Connected to a master server at 127.13.149.254:42023
I20260924 13:48:09.823879 21866 heartbeater.cc:464] Registering TS with master...
I20260924 13:48:09.824074 21866 heartbeater.cc:511] Master 127.13.149.254:42023 requested a full tablet report, sending...
I20260924 13:48:09.824465 21523 ts_manager.cc:194] Registered new tserver with Master: 74fb0a58bbc04969a2fbe9c044f837ae (127.13.149.194:46159)
I20260924 13:48:09.824844 21523 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.13.149.194@KRBTEST.COM'} at 127.13.149.194:47527
I20260924 13:48:09.828540 13911 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJN9WRF/build/release/bin/kudu as pid 21735
I20260924 13:48:09.828649 13911 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790257629041645-13911-0/minicluster-data/ts-1/wal/instance
I20260924 13:48:09.828830 13911 external_mini_cluster.cc:1428] Setting key 0aea91b198512d81dbaccbf82552e7af
I20260924 13:48:09.830252 21590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:48:09.824978 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.149.194:54011 (local address 127.13.149.254:42023)
0924 13:48:09.825019 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0924 13:48:09.825021 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:48:09.825098 (+ 77us) server_negotiation.cc:407] Connection header received
0924 13:48:09.825143 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:48:09.825143 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:48:09.825152 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:48:09.825213 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:48:09.825570 (+ 357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:48:09.826030 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:48:09.826655 (+ 625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:48:09.826824 (+ 169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:48:09.826864 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:48:09.827758 (+ 894us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:48:09.827759 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:48:09.827760 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 13:48:09.827766 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:48:09.829111 (+ 1345us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:48:09.829465 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:48:09.829466 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:48:09.829466 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:48:09.829530 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:48:09.829705 (+ 175us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:48:09.829706 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:48:09.829706 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:48:09.829856 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:48:09.829935 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0924 13:48:09.830077 (+ 142us) server_negotiation.cc:299] Negotiation successful
0924 13:48:09.830156 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260924 13:48:09.830268 21872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:48:09.824871 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:42023 (local address 127.13.149.194:54011)
0924 13:48:09.825047 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0924 13:48:09.825057 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 13:48:09.825106 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:48:09.825220 (+ 114us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:48:09.825220 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:48:09.825429 (+ 209us) client_negotiation.cc:189] Negotiated authn=SASL
0924 13:48:09.825498 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:48:09.825500 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:48:09.826108 (+ 608us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:48:09.826108 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:48:09.826597 (+ 489us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:48:09.826598 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:48:09.826872 (+ 274us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:48:09.826873 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:48:09.826970 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:48:09.827347 (+ 377us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 13:48:09.827351 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0924 13:48:09.827675 (+ 324us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 13:48:09.829182 (+ 1507us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 13:48:09.829183 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 13:48:09.829184 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 13:48:09.829395 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 13:48:09.829591 (+ 196us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 13:48:09.829591 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 13:48:09.829592 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 13:48:09.829655 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 13:48:09.829936 (+ 281us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 13:48:09.829937 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 13:48:09.829988 (+ 51us) client_negotiation.cc:769] Sending connection context
0924 13:48:09.830047 (+ 59us) client_negotiation.cc:240] Negotiation successful
0924 13:48:09.830157 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":83,"threads_started":1}
WARNING: no policy specified for oryx/127.13.149.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.13.149.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.13.149.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790257629041645-13911-0/krb5kdc/oryx_127.13.149.195.keytab.
Entry for principal oryx/127.13.149.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790257629041645-13911-0/krb5kdc/oryx_127.13.149.195.keytab.
I20260924 13:48:09.856968 13911 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJN9WRF/build/release/bin/kudu
/tmp/dist-test-taskJN9WRF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790257629041645-13911-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790257629041645-13911-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790257629041645-13911-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790257629041645-13911-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.149.195:0
--local_ip_for_outbound_sockets=127.13.149.195
--webserver_interface=127.13.149.195
--webserver_port=0
--tserver_master_addrs=127.13.149.254:42023
--keytab_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790257629041645-13911-0/krb5kdc/oryx_127.13.149.195.keytab
--principal=oryx/127.13.149.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.149.212:44865
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790257629041645-13911-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790257629041645-13911-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790257629041645-13911-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 13:48:09.942534 21875 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 13:48:09.942795 21875 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 13:48:09.942834 21875 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 13:48:09.944226 21875 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 13:48:09.944288 21875 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 13:48:09.944319 21875 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260924 13:48:09.944343 21875 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.149.195
I20260924 13:48:09.946161 21875 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.149.212:44865
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790257629041645-13911-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790257629041645-13911-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790257629041645-13911-0/krb5kdc/oryx_127.13.149.195.keytab
--principal=oryx/127.13.149.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.149.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790257629041645-13911-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.149.195
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.13.149.254:42023
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21875
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.149.195
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790257629041645-13911-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 12:29:37 UTC on ccycloud.quasar-iqzbxn.root.comops.site
build id 4670106
I20260924 13:48:09.946449 21875 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 13:48:09.946686 21875 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 13:48:09.951898 21883 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 13:48:09.951889 21881 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 13:48:09.951946 21875 server_base.cc:1034] running on GCE node
W20260924 13:48:09.951890 21880 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 13:48:09.952360 21875 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 13:48:09.952600 21875 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 13:48:09.953773 21875 hybrid_clock.cc:648] HybridClock initialized: now 1790257689953757 us; error 31 us; skew 500 ppm
Sep 24 13:48:09 dist-test-slave-sjl2 krb5kdc[21478](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257689, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.149.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 13:48:09.956429 21875 init.cc:377] Logged in from keytab as oryx/127.13.149.195@KRBTEST.COM (short username oryx)
I20260924 13:48:09.956881 21875 webserver.cc:460] Webserver started at http://127.13.149.195:40951/ using document root <none> and password file <none>
I20260924 13:48:09.957141 21875 fs_manager.cc:359] Metadata directory not provided
I20260924 13:48:09.957187 21875 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 13:48:09.957301 21875 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 13:48:09.958352 21875 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790257629041645-13911-0/minicluster-data/ts-2/data/instance:
uuid: "157ae8bbaa2c41c8b96615e2f6f5cecc"
format_stamp: "Formatted at 2026-09-24 13:48:09 on dist-test-slave-sjl2"
server_key: "1e1b514b5f00923d9cb1d7ff1900293a"
server_key_iv: "38fdc33faf1ea16767e77ea57156d9a7"
server_key_version: "encryptionkey@0"
I20260924 13:48:09.958849 21875 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790257629041645-13911-0/minicluster-data/ts-2/wal/instance:
uuid: "157ae8bbaa2c41c8b96615e2f6f5cecc"
format_stamp: "Formatted at 2026-09-24 13:48:09 on dist-test-slave-sjl2"
server_key: "1e1b514b5f00923d9cb1d7ff1900293a"
server_key_iv: "38fdc33faf1ea16767e77ea57156d9a7"
server_key_version: "encryptionkey@0"
I20260924 13:48:09.960412 21875 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260924 13:48:09.961266 21890 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 13:48:09.961408 21875 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260924 13:48:09.961493 21875 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790257629041645-13911-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790257629041645-13911-0/minicluster-data/ts-2/wal
uuid: "157ae8bbaa2c41c8b96615e2f6f5cecc"
format_stamp: "Formatted at 2026-09-24 13:48:09 on dist-test-slave-sjl2"
server_key: "1e1b514b5f00923d9cb1d7ff1900293a"
server_key_iv: "38fdc33faf1ea16767e77ea57156d9a7"
server_key_version: "encryptionkey@0"
I20260924 13:48:09.961572 21875 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790257629041645-13911-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790257629041645-13911-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790257629041645-13911-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 13:48:09.976161 21875 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 13:48:09.978204 21875 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 13:48:09.978332 21875 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 13:48:09.978965 21875 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 13:48:09.979008 21875 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 13:48:09.979032 21875 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 13:48:09.979043 21875 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 13:48:09.985910 21875 rpc_server.cc:307] RPC server started. Bound to: 127.13.149.195:46679
Sep 24 13:48:09 dist-test-slave-sjl2 krb5kdc[21478](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257689, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.149.195@KRBTEST.COM for oryx/127.13.149.254@KRBTEST.COM
I20260924 13:48:09.985983 22005 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.149.195:46679 every 8 connection(s)
I20260924 13:48:09.986325 21875 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790257629041645-13911-0/minicluster-data/ts-2/data/info.pb
I20260924 13:48:09.989073 21590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:48:09.979419 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.149.195:41599 (local address 127.13.149.254:42023)
0924 13:48:09.979493 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0924 13:48:09.979495 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:48:09.979641 (+ 146us) server_negotiation.cc:407] Connection header received
0924 13:48:09.980157 (+ 516us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:48:09.980158 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:48:09.980172 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:48:09.980250 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:48:09.981099 (+ 849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:48:09.981867 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:48:09.983416 (+ 1549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:48:09.983711 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:48:09.983794 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:48:09.986638 (+ 2844us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:48:09.986640 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:48:09.986640 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 13:48:09.986649 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:48:09.987824 (+ 1175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:48:09.988260 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:48:09.988261 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:48:09.988261 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:48:09.988317 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:48:09.988515 (+ 198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:48:09.988515 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:48:09.988515 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:48:09.988624 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:48:09.988670 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0924 13:48:09.988895 (+ 225us) server_negotiation.cc:299] Negotiation successful
0924 13:48:09.988994 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260924 13:48:09.989320 21899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:48:09.979373 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:42023 (local address 127.13.149.195:41599)
0924 13:48:09.979517 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0924 13:48:09.979558 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0924 13:48:09.980064 (+ 506us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:48:09.980319 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:48:09.980323 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:48:09.980710 (+ 387us) client_negotiation.cc:189] Negotiated authn=SASL
0924 13:48:09.981014 (+ 304us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:48:09.981020 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:48:09.981986 (+ 966us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:48:09.981987 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:48:09.983309 (+ 1322us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:48:09.983314 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:48:09.983859 (+ 545us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:48:09.983861 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:48:09.983991 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:48:09.984630 (+ 639us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 13:48:09.984642 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0924 13:48:09.986508 (+ 1866us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 13:48:09.987931 (+ 1423us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 13:48:09.987938 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 13:48:09.987940 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 13:48:09.988181 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 13:48:09.988367 (+ 186us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 13:48:09.988367 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 13:48:09.988368 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 13:48:09.988468 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 13:48:09.988668 (+ 200us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 13:48:09.988672 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 13:48:09.988795 (+ 123us) client_negotiation.cc:769] Sending connection context
0924 13:48:09.988876 (+ 81us) client_negotiation.cc:240] Negotiation successful
0924 13:48:09.989077 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":74,"threads_started":1}
I20260924 13:48:09.989840 22006 heartbeater.cc:347] Connected to a master server at 127.13.149.254:42023
I20260924 13:48:09.989926 22006 heartbeater.cc:464] Registering TS with master...
I20260924 13:48:09.990114 22006 heartbeater.cc:511] Master 127.13.149.254:42023 requested a full tablet report, sending...
I20260924 13:48:09.990482 21522 ts_manager.cc:194] Registered new tserver with Master: 157ae8bbaa2c41c8b96615e2f6f5cecc (127.13.149.195:46679)
I20260924 13:48:09.991003 21522 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.13.149.195@KRBTEST.COM'} at 127.13.149.195:41599
I20260924 13:48:09.992651 13911 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJN9WRF/build/release/bin/kudu as pid 21875
I20260924 13:48:09.992754 13911 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790257629041645-13911-0/minicluster-data/ts-2/wal/instance
I20260924 13:48:09.992969 13911 external_mini_cluster.cc:1428] Setting key 34317b61752ab817b69bfdd5332a0310
I20260924 13:48:09.993777 13911 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260924 13:48:09.996402 21590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:48:09.990941 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.149.195:33799 (local address 127.13.149.254:42023)
0924 13:48:09.990985 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0924 13:48:09.990987 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:48:09.991044 (+ 57us) server_negotiation.cc:407] Connection header received
0924 13:48:09.991102 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:48:09.991103 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:48:09.991112 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:48:09.991157 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:48:09.991510 (+ 353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:48:09.991970 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:48:09.992652 (+ 682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:48:09.992888 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:48:09.992986 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:48:09.994002 (+ 1016us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:48:09.994003 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:48:09.994003 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 13:48:09.994011 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:48:09.995363 (+ 1352us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:48:09.995683 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:48:09.995684 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:48:09.995685 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:48:09.995744 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:48:09.995908 (+ 164us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:48:09.995909 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:48:09.995909 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:48:09.996028 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:48:09.996083 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0924 13:48:09.996220 (+ 137us) server_negotiation.cc:299] Negotiation successful
0924 13:48:09.996303 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260924 13:48:09.996414 22012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:48:09.990861 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:42023 (local address 127.13.149.195:33799)
0924 13:48:09.990995 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0924 13:48:09.991004 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0924 13:48:09.991058 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:48:09.991160 (+ 102us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:48:09.991160 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:48:09.991361 (+ 201us) client_negotiation.cc:189] Negotiated authn=SASL
0924 13:48:09.991431 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:48:09.991434 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:48:09.992060 (+ 626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:48:09.992061 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:48:09.992575 (+ 514us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:48:09.992577 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:48:09.992989 (+ 412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:48:09.992990 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:48:09.993060 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:48:09.993555 (+ 495us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 13:48:09.993560 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 13:48:09.993927 (+ 367us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 13:48:09.995446 (+ 1519us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 13:48:09.995447 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 13:48:09.995448 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 13:48:09.995619 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 13:48:09.995796 (+ 177us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 13:48:09.995796 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 13:48:09.995796 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 13:48:09.995856 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 13:48:09.996079 (+ 223us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 13:48:09.996080 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 13:48:09.996132 (+ 52us) client_negotiation.cc:769] Sending connection context
0924 13:48:09.996193 (+ 61us) client_negotiation.cc:240] Negotiation successful
0924 13:48:09.996305 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":60,"threads_started":1}
Sep 24 13:48:10 dist-test-slave-sjl2 krb5kdc[21478](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257690, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 24 13:48:10 dist-test-slave-sjl2 krb5kdc[21478](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.13.149.254@KRBTEST.COM, Server not found in Kerberos database
Sep 24 13:48:10 dist-test-slave-sjl2 krb5kdc[21478](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.13.149.254@KRBTEST.COM, Server not found in Kerberos database
W20260924 13:48:10.021263 21590 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 13:48:10.012780 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46464 (local address 127.13.149.254:42023)
0924 13:48:10.012845 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0924 13:48:10.012847 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:48:10.012860 (+ 13us) server_negotiation.cc:407] Connection header received
0924 13:48:10.012957 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:48:10.012958 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:48:10.012971 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:48:10.013057 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:48:10.013830 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:48:10.014404 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:48:10.015752 (+ 1348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:48:10.016019 (+ 267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:48:10.016105 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:48:10.021173 (+ 5068us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46464: BlockingRecv error: recv got EOF from 127.0.0.1:46464 (error 108)
Metrics: {"server-negotiator.queue_time_us":32}
Sep 24 13:48:10 dist-test-slave-sjl2 krb5kdc[21478](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257690, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.13.149.254@KRBTEST.COM
I20260924 13:48:10.030591 21590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:48:10.022370 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46476 (local address 127.13.149.254:42023)
0924 13:48:10.022428 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0924 13:48:10.022431 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 13:48:10.022442 (+ 11us) server_negotiation.cc:407] Connection header received
0924 13:48:10.022513 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:48:10.022515 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:48:10.022523 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:48:10.022579 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:48:10.023141 (+ 562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:48:10.023718 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:48:10.024595 (+ 877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:48:10.024844 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:48:10.025011 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:48:10.027537 (+ 2526us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:48:10.027540 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:48:10.027540 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 13:48:10.027550 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:48:10.029025 (+ 1475us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:48:10.029503 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:48:10.029505 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:48:10.029505 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:48:10.029573 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:48:10.029800 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:48:10.029800 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:48:10.029800 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:48:10.029921 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:48:10.030001 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0924 13:48:10.030423 (+ 422us) server_negotiation.cc:299] Negotiation successful
0924 13:48:10.030514 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260924 13:48:10.032449 21522 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46476:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 13:48:10.034104 21522 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 13:48:10.041421 22030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:48:10.036821 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34888 (local address 127.13.149.193:45785)
0924 13:48:10.037401 (+ 580us) server_negotiation.cc:206] Beginning negotiation
0924 13:48:10.037405 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 13:48:10.037413 (+ 8us) server_negotiation.cc:407] Connection header received
0924 13:48:10.037449 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:48:10.037451 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:48:10.037465 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:48:10.037762 (+ 297us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 13:48:10.037817 (+ 55us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:48:10.038952 (+ 1135us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:48:10.040138 (+ 1186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:48:10.040819 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:48:10.041020 (+ 201us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:48:10.041104 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0924 13:48:10.041208 (+ 104us) server_negotiation.cc:299] Negotiation successful
0924 13:48:10.041279 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":547,"thread_start_us":120,"threads_started":1}
I20260924 13:48:10.041435 22028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:48:10.036622 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.193:45785 (local address 127.0.0.1:34888)
0924 13:48:10.036782 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0924 13:48:10.036831 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0924 13:48:10.036943 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:48:10.037642 (+ 699us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:48:10.037643 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:48:10.037644 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 13:48:10.037690 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:48:10.037691 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:48:10.039072 (+ 1381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:48:10.039074 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:48:10.040020 (+ 946us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:48:10.040024 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:48:10.040957 (+ 933us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:48:10.040958 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:48:10.041078 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:48:10.041101 (+ 23us) client_negotiation.cc:769] Sending connection context
0924 13:48:10.041190 (+ 89us) client_negotiation.cc:240] Negotiation successful
0924 13:48:10.041326 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"mutex_wait_us":65,"thread_start_us":60,"threads_started":1}
I20260924 13:48:10.041460 22032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:48:10.036820 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.195:46679 (local address 127.0.0.1:42746)
0924 13:48:10.037126 (+ 306us) negotiation.cc:107] Waiting for socket to connect
0924 13:48:10.037136 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 13:48:10.037186 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:48:10.037369 (+ 183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:48:10.037372 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:48:10.037374 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 13:48:10.037457 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:48:10.037459 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:48:10.038476 (+ 1017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:48:10.038478 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:48:10.039833 (+ 1355us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:48:10.039838 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:48:10.040968 (+ 1130us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:48:10.040969 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:48:10.041116 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:48:10.041124 (+ 8us) client_negotiation.cc:769] Sending connection context
0924 13:48:10.041186 (+ 62us) client_negotiation.cc:240] Negotiation successful
0924 13:48:10.041301 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":290,"thread_start_us":80,"threads_started":1}
I20260924 13:48:10.041661 22031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:48:10.036957 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42746 (local address 127.13.149.195:46679)
0924 13:48:10.037121 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0924 13:48:10.037126 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 13:48:10.037199 (+ 73us) server_negotiation.cc:407] Connection header received
0924 13:48:10.037244 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:48:10.037247 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:48:10.037263 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:48:10.037348 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 13:48:10.037552 (+ 204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:48:10.038389 (+ 837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:48:10.039976 (+ 1587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:48:10.040707 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:48:10.040812 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:48:10.040880 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0924 13:48:10.041512 (+ 632us) server_negotiation.cc:299] Negotiation successful
0924 13:48:10.041567 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127,"thread_start_us":78,"threads_started":1}
I20260924 13:48:10.042488 22029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:48:10.037024 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.194:46159 (local address 127.0.0.1:47830)
0924 13:48:10.037238 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0924 13:48:10.037245 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0924 13:48:10.037290 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:48:10.037546 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:48:10.037547 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:48:10.037549 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 13:48:10.037596 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:48:10.037597 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:48:10.039093 (+ 1496us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:48:10.039095 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:48:10.040264 (+ 1169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:48:10.040267 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:48:10.042192 (+ 1925us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:48:10.042194 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:48:10.042281 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:48:10.042289 (+ 8us) client_negotiation.cc:769] Sending connection context
0924 13:48:10.042346 (+ 57us) client_negotiation.cc:240] Negotiation successful
0924 13:48:10.042405 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":202,"mutex_wait_us":4,"thread_start_us":55,"threads_started":1}
I20260924 13:48:10.042850 22033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:48:10.037136 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47830 (local address 127.13.149.194:46159)
0924 13:48:10.037327 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0924 13:48:10.037332 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 13:48:10.037342 (+ 10us) server_negotiation.cc:407] Connection header received
0924 13:48:10.037383 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:48:10.037385 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:48:10.037400 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:48:10.037474 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 13:48:10.037673 (+ 199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:48:10.038915 (+ 1242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:48:10.040374 (+ 1459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:48:10.041199 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:48:10.041289 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:48:10.041350 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0924 13:48:10.042688 (+ 1338us) server_negotiation.cc:299] Negotiation successful
0924 13:48:10.042745 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":154,"thread_start_us":93,"threads_started":1}
I20260924 13:48:10.042809 21660 tablet_service.cc:1467] Processing CreateTablet for tablet 2715c695e3af4161aa9584c8bb441350 (DEFAULT_TABLE table=test-table [id=6ce99e30b39744ebb08faf69858007da]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 13:48:10.042745 21940 tablet_service.cc:1467] Processing CreateTablet for tablet 2715c695e3af4161aa9584c8bb441350 (DEFAULT_TABLE table=test-table [id=6ce99e30b39744ebb08faf69858007da]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 13:48:10.043177 21660 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2715c695e3af4161aa9584c8bb441350. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 13:48:10.043177 21940 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2715c695e3af4161aa9584c8bb441350. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 13:48:10.043464 21800 tablet_service.cc:1467] Processing CreateTablet for tablet 2715c695e3af4161aa9584c8bb441350 (DEFAULT_TABLE table=test-table [id=6ce99e30b39744ebb08faf69858007da]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 13:48:10.043792 21800 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2715c695e3af4161aa9584c8bb441350. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 13:48:10.047035 22034 tablet_bootstrap.cc:492] T 2715c695e3af4161aa9584c8bb441350 P 74fb0a58bbc04969a2fbe9c044f837ae: Bootstrap starting.
I20260924 13:48:10.047935 22034 tablet_bootstrap.cc:654] T 2715c695e3af4161aa9584c8bb441350 P 74fb0a58bbc04969a2fbe9c044f837ae: Neither blocks nor log segments found. Creating new log.
I20260924 13:48:10.048265 22034 log.cc:824] T 2715c695e3af4161aa9584c8bb441350 P 74fb0a58bbc04969a2fbe9c044f837ae: Log is configured to *not* fsync() on all Append() calls
I20260924 13:48:10.048686 22035 tablet_bootstrap.cc:492] T 2715c695e3af4161aa9584c8bb441350 P 157ae8bbaa2c41c8b96615e2f6f5cecc: Bootstrap starting.
I20260924 13:48:10.049319 22034 tablet_bootstrap.cc:492] T 2715c695e3af4161aa9584c8bb441350 P 74fb0a58bbc04969a2fbe9c044f837ae: No bootstrap required, opened a new log
I20260924 13:48:10.049341 22037 tablet_bootstrap.cc:492] T 2715c695e3af4161aa9584c8bb441350 P a66912dfe2404677a7a20ec0462d8ff0: Bootstrap starting.
I20260924 13:48:10.049417 22034 ts_tablet_manager.cc:1397] T 2715c695e3af4161aa9584c8bb441350 P 74fb0a58bbc04969a2fbe9c044f837ae: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260924 13:48:10.049455 22035 tablet_bootstrap.cc:654] T 2715c695e3af4161aa9584c8bb441350 P 157ae8bbaa2c41c8b96615e2f6f5cecc: Neither blocks nor log segments found. Creating new log.
I20260924 13:48:10.049808 22035 log.cc:824] T 2715c695e3af4161aa9584c8bb441350 P 157ae8bbaa2c41c8b96615e2f6f5cecc: Log is configured to *not* fsync() on all Append() calls
I20260924 13:48:10.049889 22037 tablet_bootstrap.cc:654] T 2715c695e3af4161aa9584c8bb441350 P a66912dfe2404677a7a20ec0462d8ff0: Neither blocks nor log segments found. Creating new log.
I20260924 13:48:10.050451 22037 log.cc:824] T 2715c695e3af4161aa9584c8bb441350 P a66912dfe2404677a7a20ec0462d8ff0: Log is configured to *not* fsync() on all Append() calls
I20260924 13:48:10.051059 22035 tablet_bootstrap.cc:492] T 2715c695e3af4161aa9584c8bb441350 P 157ae8bbaa2c41c8b96615e2f6f5cecc: No bootstrap required, opened a new log
I20260924 13:48:10.051165 22035 ts_tablet_manager.cc:1397] T 2715c695e3af4161aa9584c8bb441350 P 157ae8bbaa2c41c8b96615e2f6f5cecc: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260924 13:48:10.051537 22037 tablet_bootstrap.cc:492] T 2715c695e3af4161aa9584c8bb441350 P a66912dfe2404677a7a20ec0462d8ff0: No bootstrap required, opened a new log
I20260924 13:48:10.051615 22037 ts_tablet_manager.cc:1397] T 2715c695e3af4161aa9584c8bb441350 P a66912dfe2404677a7a20ec0462d8ff0: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260924 13:48:10.051739 22034 raft_consensus.cc:348] T 2715c695e3af4161aa9584c8bb441350 P 74fb0a58bbc04969a2fbe9c044f837ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a66912dfe2404677a7a20ec0462d8ff0" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 45785 } } peers { permanent_uuid: "157ae8bbaa2c41c8b96615e2f6f5cecc" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 46679 } } peers { permanent_uuid: "74fb0a58bbc04969a2fbe9c044f837ae" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 46159 } }
I20260924 13:48:10.051954 22034 raft_consensus.cc:374] T 2715c695e3af4161aa9584c8bb441350 P 74fb0a58bbc04969a2fbe9c044f837ae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 13:48:10.052004 22034 raft_consensus.cc:729] T 2715c695e3af4161aa9584c8bb441350 P 74fb0a58bbc04969a2fbe9c044f837ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 74fb0a58bbc04969a2fbe9c044f837ae, State: Initialized, Role: FOLLOWER
I20260924 13:48:10.052173 22034 consensus_queue.cc:260] T 2715c695e3af4161aa9584c8bb441350 P 74fb0a58bbc04969a2fbe9c044f837ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a66912dfe2404677a7a20ec0462d8ff0" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 45785 } } peers { permanent_uuid: "157ae8bbaa2c41c8b96615e2f6f5cecc" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 46679 } } peers { permanent_uuid: "74fb0a58bbc04969a2fbe9c044f837ae" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 46159 } }
I20260924 13:48:10.052501 22034 ts_tablet_manager.cc:1428] T 2715c695e3af4161aa9584c8bb441350 P 74fb0a58bbc04969a2fbe9c044f837ae: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260924 13:48:10.052547 21866 heartbeater.cc:503] Master 127.13.149.254:42023 was elected leader, sending a full tablet report...
I20260924 13:48:10.053459 22035 raft_consensus.cc:348] T 2715c695e3af4161aa9584c8bb441350 P 157ae8bbaa2c41c8b96615e2f6f5cecc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a66912dfe2404677a7a20ec0462d8ff0" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 45785 } } peers { permanent_uuid: "157ae8bbaa2c41c8b96615e2f6f5cecc" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 46679 } } peers { permanent_uuid: "74fb0a58bbc04969a2fbe9c044f837ae" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 46159 } }
I20260924 13:48:10.053660 22035 raft_consensus.cc:374] T 2715c695e3af4161aa9584c8bb441350 P 157ae8bbaa2c41c8b96615e2f6f5cecc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 13:48:10.053705 22035 raft_consensus.cc:729] T 2715c695e3af4161aa9584c8bb441350 P 157ae8bbaa2c41c8b96615e2f6f5cecc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 157ae8bbaa2c41c8b96615e2f6f5cecc, State: Initialized, Role: FOLLOWER
I20260924 13:48:10.053854 22035 consensus_queue.cc:260] T 2715c695e3af4161aa9584c8bb441350 P 157ae8bbaa2c41c8b96615e2f6f5cecc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a66912dfe2404677a7a20ec0462d8ff0" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 45785 } } peers { permanent_uuid: "157ae8bbaa2c41c8b96615e2f6f5cecc" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 46679 } } peers { permanent_uuid: "74fb0a58bbc04969a2fbe9c044f837ae" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 46159 } }
I20260924 13:48:10.054183 22035 ts_tablet_manager.cc:1428] T 2715c695e3af4161aa9584c8bb441350 P 157ae8bbaa2c41c8b96615e2f6f5cecc: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260924 13:48:10.054279 22037 raft_consensus.cc:348] T 2715c695e3af4161aa9584c8bb441350 P a66912dfe2404677a7a20ec0462d8ff0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a66912dfe2404677a7a20ec0462d8ff0" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 45785 } } peers { permanent_uuid: "157ae8bbaa2c41c8b96615e2f6f5cecc" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 46679 } } peers { permanent_uuid: "74fb0a58bbc04969a2fbe9c044f837ae" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 46159 } }
I20260924 13:48:10.054441 22006 heartbeater.cc:503] Master 127.13.149.254:42023 was elected leader, sending a full tablet report...
I20260924 13:48:10.054473 22037 raft_consensus.cc:374] T 2715c695e3af4161aa9584c8bb441350 P a66912dfe2404677a7a20ec0462d8ff0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 13:48:10.054510 22037 raft_consensus.cc:729] T 2715c695e3af4161aa9584c8bb441350 P a66912dfe2404677a7a20ec0462d8ff0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a66912dfe2404677a7a20ec0462d8ff0, State: Initialized, Role: FOLLOWER
I20260924 13:48:10.054673 22037 consensus_queue.cc:260] T 2715c695e3af4161aa9584c8bb441350 P a66912dfe2404677a7a20ec0462d8ff0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a66912dfe2404677a7a20ec0462d8ff0" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 45785 } } peers { permanent_uuid: "157ae8bbaa2c41c8b96615e2f6f5cecc" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 46679 } } peers { permanent_uuid: "74fb0a58bbc04969a2fbe9c044f837ae" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 46159 } }
I20260924 13:48:10.055027 21726 heartbeater.cc:503] Master 127.13.149.254:42023 was elected leader, sending a full tablet report...
I20260924 13:48:10.055279 22037 ts_tablet_manager.cc:1428] T 2715c695e3af4161aa9584c8bb441350 P a66912dfe2404677a7a20ec0462d8ff0: Time spent starting tablet: real 0.004s user 0.000s sys 0.004s
I20260924 13:48:10.063664 22042 raft_consensus.cc:482] T 2715c695e3af4161aa9584c8bb441350 P a66912dfe2404677a7a20ec0462d8ff0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 13:48:10.063819 22042 raft_consensus.cc:504] T 2715c695e3af4161aa9584c8bb441350 P a66912dfe2404677a7a20ec0462d8ff0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a66912dfe2404677a7a20ec0462d8ff0" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 45785 } } peers { permanent_uuid: "157ae8bbaa2c41c8b96615e2f6f5cecc" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 46679 } } peers { permanent_uuid: "74fb0a58bbc04969a2fbe9c044f837ae" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 46159 } }
I20260924 13:48:10.064139 22042 leader_election.cc:288] T 2715c695e3af4161aa9584c8bb441350 P a66912dfe2404677a7a20ec0462d8ff0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 157ae8bbaa2c41c8b96615e2f6f5cecc (127.13.149.195:46679), 74fb0a58bbc04969a2fbe9c044f837ae (127.13.149.194:46159)
I20260924 13:48:10.067480 22031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:48:10.064523 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.149.193:37273 (local address 127.13.149.195:46679)
0924 13:48:10.064691 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0924 13:48:10.064694 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 13:48:10.064707 (+ 13us) server_negotiation.cc:407] Connection header received
0924 13:48:10.064740 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:48:10.064740 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:48:10.064750 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:48:10.064826 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 13:48:10.065042 (+ 216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:48:10.065574 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:48:10.066756 (+ 1182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:48:10.067119 (+ 363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:48:10.067185 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:48:10.067194 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0924 13:48:10.067352 (+ 158us) server_negotiation.cc:299] Negotiation successful
0924 13:48:10.067391 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":147}
I20260924 13:48:10.067484 21619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:48:10.064371 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.195:46679 (local address 127.13.149.193:37273)
0924 13:48:10.064459 (+ 88us) negotiation.cc:107] Waiting for socket to connect
0924 13:48:10.064476 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0924 13:48:10.064587 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:48:10.064846 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:48:10.064847 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:48:10.064849 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 13:48:10.064970 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:48:10.064972 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:48:10.065702 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:48:10.065703 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:48:10.066656 (+ 953us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:48:10.066659 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:48:10.067193 (+ 534us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:48:10.067194 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:48:10.067288 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:48:10.067292 (+ 4us) client_negotiation.cc:769] Sending connection context
0924 13:48:10.067339 (+ 47us) client_negotiation.cc:240] Negotiation successful
0924 13:48:10.067385 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":46}
I20260924 13:48:10.067571 22033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:48:10.064714 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.149.193:41685 (local address 127.13.149.194:46159)
0924 13:48:10.064807 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0924 13:48:10.064809 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:48:10.064821 (+ 12us) server_negotiation.cc:407] Connection header received
0924 13:48:10.064846 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:48:10.064847 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:48:10.064857 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:48:10.064993 (+ 136us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 13:48:10.065177 (+ 184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:48:10.065865 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:48:10.066867 (+ 1002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:48:10.067233 (+ 366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:48:10.067299 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:48:10.067308 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0924 13:48:10.067445 (+ 137us) server_negotiation.cc:299] Negotiation successful
0924 13:48:10.067493 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260924 13:48:10.067617 22043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:48:10.064468 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.194:46159 (local address 127.13.149.193:41685)
0924 13:48:10.064661 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0924 13:48:10.064675 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0924 13:48:10.064754 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:48:10.065005 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:48:10.065005 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:48:10.065007 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 13:48:10.065094 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:48:10.065096 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:48:10.065956 (+ 860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:48:10.065957 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:48:10.066774 (+ 817us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:48:10.066777 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:48:10.067307 (+ 530us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:48:10.067308 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:48:10.067384 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:48:10.067388 (+ 4us) client_negotiation.cc:769] Sending connection context
0924 13:48:10.067433 (+ 45us) client_negotiation.cc:240] Negotiation successful
0924 13:48:10.067479 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":167,"mutex_wait_us":14,"thread_start_us":132,"threads_started":1}
I20260924 13:48:10.067857 21960 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2715c695e3af4161aa9584c8bb441350" candidate_uuid: "a66912dfe2404677a7a20ec0462d8ff0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "157ae8bbaa2c41c8b96615e2f6f5cecc" is_pre_election: true
I20260924 13:48:10.067896 21820 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2715c695e3af4161aa9584c8bb441350" candidate_uuid: "a66912dfe2404677a7a20ec0462d8ff0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74fb0a58bbc04969a2fbe9c044f837ae" is_pre_election: true
I20260924 13:48:10.068001 21960 raft_consensus.cc:2454] T 2715c695e3af4161aa9584c8bb441350 P 157ae8bbaa2c41c8b96615e2f6f5cecc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a66912dfe2404677a7a20ec0462d8ff0 in term 0.
I20260924 13:48:10.068050 21820 raft_consensus.cc:2454] T 2715c695e3af4161aa9584c8bb441350 P 74fb0a58bbc04969a2fbe9c044f837ae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a66912dfe2404677a7a20ec0462d8ff0 in term 0.
I20260924 13:48:10.068253 21611 leader_election.cc:302] T 2715c695e3af4161aa9584c8bb441350 P a66912dfe2404677a7a20ec0462d8ff0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 157ae8bbaa2c41c8b96615e2f6f5cecc, a66912dfe2404677a7a20ec0462d8ff0; no voters:
I20260924 13:48:10.068387 22042 raft_consensus.cc:2781] T 2715c695e3af4161aa9584c8bb441350 P a66912dfe2404677a7a20ec0462d8ff0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 13:48:10.068445 22042 raft_consensus.cc:482] T 2715c695e3af4161aa9584c8bb441350 P a66912dfe2404677a7a20ec0462d8ff0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 13:48:10.068477 22042 raft_consensus.cc:3037] T 2715c695e3af4161aa9584c8bb441350 P a66912dfe2404677a7a20ec0462d8ff0 [term 0 FOLLOWER]: Advancing to term 1
I20260924 13:48:10.069532 22042 raft_consensus.cc:504] T 2715c695e3af4161aa9584c8bb441350 P a66912dfe2404677a7a20ec0462d8ff0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a66912dfe2404677a7a20ec0462d8ff0" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 45785 } } peers { permanent_uuid: "157ae8bbaa2c41c8b96615e2f6f5cecc" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 46679 } } peers { permanent_uuid: "74fb0a58bbc04969a2fbe9c044f837ae" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 46159 } }
I20260924 13:48:10.069669 22042 leader_election.cc:288] T 2715c695e3af4161aa9584c8bb441350 P a66912dfe2404677a7a20ec0462d8ff0 [CANDIDATE]: Term 1 election: Requested vote from peers 157ae8bbaa2c41c8b96615e2f6f5cecc (127.13.149.195:46679), 74fb0a58bbc04969a2fbe9c044f837ae (127.13.149.194:46159)
I20260924 13:48:10.069901 21820 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2715c695e3af4161aa9584c8bb441350" candidate_uuid: "a66912dfe2404677a7a20ec0462d8ff0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74fb0a58bbc04969a2fbe9c044f837ae"
I20260924 13:48:10.069898 21960 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2715c695e3af4161aa9584c8bb441350" candidate_uuid: "a66912dfe2404677a7a20ec0462d8ff0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "157ae8bbaa2c41c8b96615e2f6f5cecc"
I20260924 13:48:10.069984 21820 raft_consensus.cc:3037] T 2715c695e3af4161aa9584c8bb441350 P 74fb0a58bbc04969a2fbe9c044f837ae [term 0 FOLLOWER]: Advancing to term 1
I20260924 13:48:10.069995 21960 raft_consensus.cc:3037] T 2715c695e3af4161aa9584c8bb441350 P 157ae8bbaa2c41c8b96615e2f6f5cecc [term 0 FOLLOWER]: Advancing to term 1
I20260924 13:48:10.070747 21820 raft_consensus.cc:2454] T 2715c695e3af4161aa9584c8bb441350 P 74fb0a58bbc04969a2fbe9c044f837ae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a66912dfe2404677a7a20ec0462d8ff0 in term 1.
I20260924 13:48:10.070781 21960 raft_consensus.cc:2454] T 2715c695e3af4161aa9584c8bb441350 P 157ae8bbaa2c41c8b96615e2f6f5cecc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a66912dfe2404677a7a20ec0462d8ff0 in term 1.
I20260924 13:48:10.070925 21611 leader_election.cc:302] T 2715c695e3af4161aa9584c8bb441350 P a66912dfe2404677a7a20ec0462d8ff0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74fb0a58bbc04969a2fbe9c044f837ae, a66912dfe2404677a7a20ec0462d8ff0; no voters:
I20260924 13:48:10.071040 22042 raft_consensus.cc:2781] T 2715c695e3af4161aa9584c8bb441350 P a66912dfe2404677a7a20ec0462d8ff0 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 13:48:10.071223 22042 raft_consensus.cc:686] T 2715c695e3af4161aa9584c8bb441350 P a66912dfe2404677a7a20ec0462d8ff0 [term 1 LEADER]: Becoming Leader. State: Replica: a66912dfe2404677a7a20ec0462d8ff0, State: Running, Role: LEADER
I20260924 13:48:10.071329 22042 consensus_queue.cc:237] T 2715c695e3af4161aa9584c8bb441350 P a66912dfe2404677a7a20ec0462d8ff0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a66912dfe2404677a7a20ec0462d8ff0" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 45785 } } peers { permanent_uuid: "157ae8bbaa2c41c8b96615e2f6f5cecc" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 46679 } } peers { permanent_uuid: "74fb0a58bbc04969a2fbe9c044f837ae" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 46159 } }
W20260924 13:48:10.071466 21867 tablet.cc:2369] T 2715c695e3af4161aa9584c8bb441350 P 74fb0a58bbc04969a2fbe9c044f837ae: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260924 13:48:10.072330 21523 catalog_manager.cc:5495] T 2715c695e3af4161aa9584c8bb441350 P a66912dfe2404677a7a20ec0462d8ff0 reported cstate change: term changed from 0 to 1, leader changed from <none> to a66912dfe2404677a7a20ec0462d8ff0 (127.13.149.193). New cstate: current_term: 1 leader_uuid: "a66912dfe2404677a7a20ec0462d8ff0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a66912dfe2404677a7a20ec0462d8ff0" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 45785 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "157ae8bbaa2c41c8b96615e2f6f5cecc" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 46679 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "74fb0a58bbc04969a2fbe9c044f837ae" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 46159 } health_report { overall_health: UNKNOWN } } }
I20260924 13:48:10.079069 22030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:48:10.076262 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34890 (local address 127.13.149.193:45785)
0924 13:48:10.076331 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0924 13:48:10.076333 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:48:10.076342 (+ 9us) server_negotiation.cc:407] Connection header received
0924 13:48:10.076377 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:48:10.076378 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:48:10.076395 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:48:10.076455 (+ 60us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 13:48:10.076604 (+ 149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:48:10.077114 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:48:10.077949 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:48:10.078205 (+ 256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:48:10.078296 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:48:10.078567 (+ 271us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 13:48:10.078694 (+ 127us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 13:48:10.078778 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0924 13:48:10.078857 (+ 79us) server_negotiation.cc:299] Negotiation successful
0924 13:48:10.078900 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260924 13:48:10.081121 21820 raft_consensus.cc:1261] T 2715c695e3af4161aa9584c8bb441350 P 74fb0a58bbc04969a2fbe9c044f837ae [term 1 FOLLOWER]: Refusing update from remote peer a66912dfe2404677a7a20ec0462d8ff0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 13:48:10.081173 21960 raft_consensus.cc:1261] T 2715c695e3af4161aa9584c8bb441350 P 157ae8bbaa2c41c8b96615e2f6f5cecc [term 1 FOLLOWER]: Refusing update from remote peer a66912dfe2404677a7a20ec0462d8ff0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 13:48:10.081497 22044 consensus_queue.cc:1035] T 2715c695e3af4161aa9584c8bb441350 P a66912dfe2404677a7a20ec0462d8ff0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "157ae8bbaa2c41c8b96615e2f6f5cecc" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 46679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 13:48:10.081689 22042 consensus_queue.cc:1035] T 2715c695e3af4161aa9584c8bb441350 P a66912dfe2404677a7a20ec0462d8ff0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "74fb0a58bbc04969a2fbe9c044f837ae" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 46159 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 13:48:10.089637 21522 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46476:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 13:48:10.089782 21522 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46476:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 13:48:10.090642 21522 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9f6c1d2f544b490eb7522be8ff50ceba: Sending DeleteTablet for 3 replicas of tablet 2715c695e3af4161aa9584c8bb441350
I20260924 13:48:10.091084 21940 tablet_service.cc:1514] Processing DeleteTablet for tablet 2715c695e3af4161aa9584c8bb441350 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 13:48:10 UTC) from {username='oryx', principal='oryx/127.13.149.254@KRBTEST.COM'} at 127.0.0.1:42746
I20260924 13:48:10.091084 21660 tablet_service.cc:1514] Processing DeleteTablet for tablet 2715c695e3af4161aa9584c8bb441350 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 13:48:10 UTC) from {username='oryx', principal='oryx/127.13.149.254@KRBTEST.COM'} at 127.0.0.1:34888
I20260924 13:48:10.091262 21522 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46476:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 13:48:10.091424 21522 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 13:48:10.091447 21800 tablet_service.cc:1514] Processing DeleteTablet for tablet 2715c695e3af4161aa9584c8bb441350 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 13:48:10 UTC) from {username='oryx', principal='oryx/127.13.149.254@KRBTEST.COM'} at 127.0.0.1:47830
I20260924 13:48:10.091677 22058 tablet_replica.cc:330] T 2715c695e3af4161aa9584c8bb441350 P 157ae8bbaa2c41c8b96615e2f6f5cecc: stopping tablet replica
I20260924 13:48:10.091706 22057 tablet_replica.cc:330] T 2715c695e3af4161aa9584c8bb441350 P a66912dfe2404677a7a20ec0462d8ff0: stopping tablet replica
I20260924 13:48:10.091828 22057 raft_consensus.cc:2229] T 2715c695e3af4161aa9584c8bb441350 P a66912dfe2404677a7a20ec0462d8ff0 [term 1 LEADER]: Raft consensus shutting down.
I20260924 13:48:10.091849 22058 raft_consensus.cc:2229] T 2715c695e3af4161aa9584c8bb441350 P 157ae8bbaa2c41c8b96615e2f6f5cecc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 13:48:10.091984 22057 raft_consensus.cc:2258] T 2715c695e3af4161aa9584c8bb441350 P a66912dfe2404677a7a20ec0462d8ff0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 13:48:10.092114 22058 raft_consensus.cc:2258] T 2715c695e3af4161aa9584c8bb441350 P 157ae8bbaa2c41c8b96615e2f6f5cecc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 13:48:10.092762 21940 tablet_service.cc:1467] Processing CreateTablet for tablet 47e5d842ec0042598a651c906036debc (DEFAULT_TABLE table=test-table [id=e78a1ae8505147b2bd6f123195763ec8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 13:48:10.092762 21660 tablet_service.cc:1467] Processing CreateTablet for tablet 47e5d842ec0042598a651c906036debc (DEFAULT_TABLE table=test-table [id=e78a1ae8505147b2bd6f123195763ec8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 13:48:10.092882 21940 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 47e5d842ec0042598a651c906036debc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 13:48:10.092882 21660 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 47e5d842ec0042598a651c906036debc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 13:48:10.093034 21800 tablet_service.cc:1467] Processing CreateTablet for tablet 47e5d842ec0042598a651c906036debc (DEFAULT_TABLE table=test-table [id=e78a1ae8505147b2bd6f123195763ec8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 13:48:10.093137 21800 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 47e5d842ec0042598a651c906036debc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 13:48:10.094594 22035 tablet_bootstrap.cc:492] T 47e5d842ec0042598a651c906036debc P 157ae8bbaa2c41c8b96615e2f6f5cecc: Bootstrap starting.
I20260924 13:48:10.094830 22034 tablet_bootstrap.cc:492] T 47e5d842ec0042598a651c906036debc P 74fb0a58bbc04969a2fbe9c044f837ae: Bootstrap starting.
I20260924 13:48:10.095038 22035 tablet_bootstrap.cc:654] T 47e5d842ec0042598a651c906036debc P 157ae8bbaa2c41c8b96615e2f6f5cecc: Neither blocks nor log segments found. Creating new log.
I20260924 13:48:10.095310 22034 tablet_bootstrap.cc:654] T 47e5d842ec0042598a651c906036debc P 74fb0a58bbc04969a2fbe9c044f837ae: Neither blocks nor log segments found. Creating new log.
I20260924 13:48:10.092617 22058 ts_tablet_manager.cc:1905] T 2715c695e3af4161aa9584c8bb441350 P 157ae8bbaa2c41c8b96615e2f6f5cecc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 13:48:10.096309 22037 tablet_bootstrap.cc:492] T 47e5d842ec0042598a651c906036debc P a66912dfe2404677a7a20ec0462d8ff0: Bootstrap starting.
I20260924 13:48:10.096603 22059 tablet_replica.cc:330] T 2715c695e3af4161aa9584c8bb441350 P 74fb0a58bbc04969a2fbe9c044f837ae: stopping tablet replica
I20260924 13:48:10.096730 22059 raft_consensus.cc:2229] T 2715c695e3af4161aa9584c8bb441350 P 74fb0a58bbc04969a2fbe9c044f837ae [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 13:48:10.096872 22059 raft_consensus.cc:2258] T 2715c695e3af4161aa9584c8bb441350 P 74fb0a58bbc04969a2fbe9c044f837ae [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 13:48:10.096462 22057 ts_tablet_manager.cc:1905] T 2715c695e3af4161aa9584c8bb441350 P a66912dfe2404677a7a20ec0462d8ff0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 13:48:10.097157 22058 ts_tablet_manager.cc:1918] T 2715c695e3af4161aa9584c8bb441350 P 157ae8bbaa2c41c8b96615e2f6f5cecc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 13:48:10.097239 22058 log.cc:1196] T 2715c695e3af4161aa9584c8bb441350 P 157ae8bbaa2c41c8b96615e2f6f5cecc: Deleting WAL directory at /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790257629041645-13911-0/minicluster-data/ts-2/wal/wals/2715c695e3af4161aa9584c8bb441350
I20260924 13:48:10.097469 22037 tablet_bootstrap.cc:654] T 47e5d842ec0042598a651c906036debc P a66912dfe2404677a7a20ec0462d8ff0: Neither blocks nor log segments found. Creating new log.
I20260924 13:48:10.097497 22058 ts_tablet_manager.cc:1939] T 2715c695e3af4161aa9584c8bb441350 P 157ae8bbaa2c41c8b96615e2f6f5cecc: Deleting consensus metadata
I20260924 13:48:10.097944 21507 catalog_manager.cc:4841] TS 157ae8bbaa2c41c8b96615e2f6f5cecc (127.13.149.195:46679): tablet 2715c695e3af4161aa9584c8bb441350 (table test-table [id=6ce99e30b39744ebb08faf69858007da]) successfully deleted
I20260924 13:48:10.098462 22035 tablet_bootstrap.cc:492] T 47e5d842ec0042598a651c906036debc P 157ae8bbaa2c41c8b96615e2f6f5cecc: No bootstrap required, opened a new log
I20260924 13:48:10.098495 22057 ts_tablet_manager.cc:1918] T 2715c695e3af4161aa9584c8bb441350 P a66912dfe2404677a7a20ec0462d8ff0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 13:48:10.098523 22035 ts_tablet_manager.cc:1397] T 47e5d842ec0042598a651c906036debc P 157ae8bbaa2c41c8b96615e2f6f5cecc: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260924 13:48:10.098559 22057 log.cc:1196] T 2715c695e3af4161aa9584c8bb441350 P a66912dfe2404677a7a20ec0462d8ff0: Deleting WAL directory at /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790257629041645-13911-0/minicluster-data/ts-0/wal/wals/2715c695e3af4161aa9584c8bb441350
I20260924 13:48:10.098672 22035 raft_consensus.cc:348] T 47e5d842ec0042598a651c906036debc P 157ae8bbaa2c41c8b96615e2f6f5cecc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a66912dfe2404677a7a20ec0462d8ff0" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 45785 } } peers { permanent_uuid: "74fb0a58bbc04969a2fbe9c044f837ae" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 46159 } } peers { permanent_uuid: "157ae8bbaa2c41c8b96615e2f6f5cecc" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 46679 } }
I20260924 13:48:10.098774 22035 raft_consensus.cc:374] T 47e5d842ec0042598a651c906036debc P 157ae8bbaa2c41c8b96615e2f6f5cecc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 13:48:10.098793 22057 ts_tablet_manager.cc:1939] T 2715c695e3af4161aa9584c8bb441350 P a66912dfe2404677a7a20ec0462d8ff0: Deleting consensus metadata
I20260924 13:48:10.098801 22035 raft_consensus.cc:729] T 47e5d842ec0042598a651c906036debc P 157ae8bbaa2c41c8b96615e2f6f5cecc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 157ae8bbaa2c41c8b96615e2f6f5cecc, State: Initialized, Role: FOLLOWER
I20260924 13:48:10.098861 22035 consensus_queue.cc:260] T 47e5d842ec0042598a651c906036debc P 157ae8bbaa2c41c8b96615e2f6f5cecc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a66912dfe2404677a7a20ec0462d8ff0" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 45785 } } peers { permanent_uuid: "74fb0a58bbc04969a2fbe9c044f837ae" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 46159 } } peers { permanent_uuid: "157ae8bbaa2c41c8b96615e2f6f5cecc" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 46679 } }
I20260924 13:48:10.098977 22035 ts_tablet_manager.cc:1428] T 47e5d842ec0042598a651c906036debc P 157ae8bbaa2c41c8b96615e2f6f5cecc: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260924 13:48:10.099182 21507 catalog_manager.cc:4841] TS a66912dfe2404677a7a20ec0462d8ff0 (127.13.149.193:45785): tablet 2715c695e3af4161aa9584c8bb441350 (table test-table [id=6ce99e30b39744ebb08faf69858007da]) successfully deleted
I20260924 13:48:10.099720 22059 ts_tablet_manager.cc:1905] T 2715c695e3af4161aa9584c8bb441350 P 74fb0a58bbc04969a2fbe9c044f837ae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 13:48:10.101106 22059 ts_tablet_manager.cc:1918] T 2715c695e3af4161aa9584c8bb441350 P 74fb0a58bbc04969a2fbe9c044f837ae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 13:48:10.101186 22059 log.cc:1196] T 2715c695e3af4161aa9584c8bb441350 P 74fb0a58bbc04969a2fbe9c044f837ae: Deleting WAL directory at /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790257629041645-13911-0/minicluster-data/ts-1/wal/wals/2715c695e3af4161aa9584c8bb441350
I20260924 13:48:10.102794 22059 ts_tablet_manager.cc:1939] T 2715c695e3af4161aa9584c8bb441350 P 74fb0a58bbc04969a2fbe9c044f837ae: Deleting consensus metadata
I20260924 13:48:10.103282 21507 catalog_manager.cc:4841] TS 74fb0a58bbc04969a2fbe9c044f837ae (127.13.149.194:46159): tablet 2715c695e3af4161aa9584c8bb441350 (table test-table [id=6ce99e30b39744ebb08faf69858007da]) successfully deleted
I20260924 13:48:10.105422 22034 tablet_bootstrap.cc:492] T 47e5d842ec0042598a651c906036debc P 74fb0a58bbc04969a2fbe9c044f837ae: No bootstrap required, opened a new log
I20260924 13:48:10.105489 22034 ts_tablet_manager.cc:1397] T 47e5d842ec0042598a651c906036debc P 74fb0a58bbc04969a2fbe9c044f837ae: Time spent bootstrapping tablet: real 0.011s user 0.001s sys 0.000s
I20260924 13:48:10.105652 22034 raft_consensus.cc:348] T 47e5d842ec0042598a651c906036debc P 74fb0a58bbc04969a2fbe9c044f837ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a66912dfe2404677a7a20ec0462d8ff0" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 45785 } } peers { permanent_uuid: "74fb0a58bbc04969a2fbe9c044f837ae" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 46159 } } peers { permanent_uuid: "157ae8bbaa2c41c8b96615e2f6f5cecc" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 46679 } }
I20260924 13:48:10.105744 22034 raft_consensus.cc:374] T 47e5d842ec0042598a651c906036debc P 74fb0a58bbc04969a2fbe9c044f837ae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 13:48:10.105772 22034 raft_consensus.cc:729] T 47e5d842ec0042598a651c906036debc P 74fb0a58bbc04969a2fbe9c044f837ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 74fb0a58bbc04969a2fbe9c044f837ae, State: Initialized, Role: FOLLOWER
I20260924 13:48:10.105822 22034 consensus_queue.cc:260] T 47e5d842ec0042598a651c906036debc P 74fb0a58bbc04969a2fbe9c044f837ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a66912dfe2404677a7a20ec0462d8ff0" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 45785 } } peers { permanent_uuid: "74fb0a58bbc04969a2fbe9c044f837ae" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 46159 } } peers { permanent_uuid: "157ae8bbaa2c41c8b96615e2f6f5cecc" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 46679 } }
I20260924 13:48:10.105917 22034 ts_tablet_manager.cc:1428] T 47e5d842ec0042598a651c906036debc P 74fb0a58bbc04969a2fbe9c044f837ae: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260924 13:48:10.105909 22037 tablet_bootstrap.cc:492] T 47e5d842ec0042598a651c906036debc P a66912dfe2404677a7a20ec0462d8ff0: No bootstrap required, opened a new log
I20260924 13:48:10.106006 22037 ts_tablet_manager.cc:1397] T 47e5d842ec0042598a651c906036debc P a66912dfe2404677a7a20ec0462d8ff0: Time spent bootstrapping tablet: real 0.010s user 0.000s sys 0.001s
I20260924 13:48:10.106186 22037 raft_consensus.cc:348] T 47e5d842ec0042598a651c906036debc P a66912dfe2404677a7a20ec0462d8ff0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a66912dfe2404677a7a20ec0462d8ff0" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 45785 } } peers { permanent_uuid: "74fb0a58bbc04969a2fbe9c044f837ae" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 46159 } } peers { permanent_uuid: "157ae8bbaa2c41c8b96615e2f6f5cecc" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 46679 } }
I20260924 13:48:10.106281 22037 raft_consensus.cc:374] T 47e5d842ec0042598a651c906036debc P a66912dfe2404677a7a20ec0462d8ff0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 13:48:10.106318 22037 raft_consensus.cc:729] T 47e5d842ec0042598a651c906036debc P a66912dfe2404677a7a20ec0462d8ff0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a66912dfe2404677a7a20ec0462d8ff0, State: Initialized, Role: FOLLOWER
I20260924 13:48:10.106390 22037 consensus_queue.cc:260] T 47e5d842ec0042598a651c906036debc P a66912dfe2404677a7a20ec0462d8ff0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a66912dfe2404677a7a20ec0462d8ff0" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 45785 } } peers { permanent_uuid: "74fb0a58bbc04969a2fbe9c044f837ae" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 46159 } } peers { permanent_uuid: "157ae8bbaa2c41c8b96615e2f6f5cecc" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 46679 } }
I20260924 13:48:10.106499 22037 ts_tablet_manager.cc:1428] T 47e5d842ec0042598a651c906036debc P a66912dfe2404677a7a20ec0462d8ff0: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
W20260924 13:48:10.156387 21727 tablet.cc:2369] T 47e5d842ec0042598a651c906036debc P a66912dfe2404677a7a20ec0462d8ff0: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260924 13:48:10.237063 22007 tablet.cc:2369] T 47e5d842ec0042598a651c906036debc P 157ae8bbaa2c41c8b96615e2f6f5cecc: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260924 13:48:10.240479 22041 raft_consensus.cc:482] T 47e5d842ec0042598a651c906036debc P 157ae8bbaa2c41c8b96615e2f6f5cecc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 13:48:10.240584 22041 raft_consensus.cc:504] T 47e5d842ec0042598a651c906036debc P 157ae8bbaa2c41c8b96615e2f6f5cecc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a66912dfe2404677a7a20ec0462d8ff0" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 45785 } } peers { permanent_uuid: "74fb0a58bbc04969a2fbe9c044f837ae" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 46159 } } peers { permanent_uuid: "157ae8bbaa2c41c8b96615e2f6f5cecc" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 46679 } }
I20260924 13:48:10.240903 22041 leader_election.cc:288] T 47e5d842ec0042598a651c906036debc P 157ae8bbaa2c41c8b96615e2f6f5cecc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a66912dfe2404677a7a20ec0462d8ff0 (127.13.149.193:45785), 74fb0a58bbc04969a2fbe9c044f837ae (127.13.149.194:46159)
I20260924 13:48:10.244969 22033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:48:10.241272 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.149.195:45305 (local address 127.13.149.194:46159)
0924 13:48:10.241319 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0924 13:48:10.241321 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:48:10.241447 (+ 126us) server_negotiation.cc:407] Connection header received
0924 13:48:10.241469 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:48:10.241470 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:48:10.241477 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:48:10.241524 (+ 47us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 13:48:10.241645 (+ 121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:48:10.242322 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:48:10.243883 (+ 1561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:48:10.244469 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:48:10.244557 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:48:10.244568 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0924 13:48:10.244803 (+ 235us) server_negotiation.cc:299] Negotiation successful
0924 13:48:10.244855 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260924 13:48:10.244961 22030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:48:10.241181 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.149.195:60709 (local address 127.13.149.193:45785)
0924 13:48:10.241241 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0924 13:48:10.241243 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:48:10.241251 (+ 8us) server_negotiation.cc:407] Connection header received
0924 13:48:10.241291 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:48:10.241292 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:48:10.241303 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:48:10.241351 (+ 48us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 13:48:10.241525 (+ 174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:48:10.242283 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:48:10.243883 (+ 1600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:48:10.244469 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:48:10.244583 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:48:10.244591 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0924 13:48:10.244802 (+ 211us) server_negotiation.cc:299] Negotiation successful
0924 13:48:10.244854 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260924 13:48:10.244946 21899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:48:10.241101 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.193:45785 (local address 127.13.149.195:60709)
0924 13:48:10.241154 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0924 13:48:10.241168 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0924 13:48:10.241228 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:48:10.241373 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:48:10.241374 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:48:10.241376 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 13:48:10.241455 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:48:10.241457 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:48:10.242382 (+ 925us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:48:10.242384 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:48:10.243739 (+ 1355us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:48:10.243744 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:48:10.244586 (+ 842us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:48:10.244587 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:48:10.244691 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:48:10.244695 (+ 4us) client_negotiation.cc:769] Sending connection context
0924 13:48:10.244758 (+ 63us) client_negotiation.cc:240] Negotiation successful
0924 13:48:10.244814 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30,"mutex_wait_us":48}
I20260924 13:48:10.244956 22063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:48:10.241200 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.194:46159 (local address 127.13.149.195:45305)
0924 13:48:10.241340 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0924 13:48:10.241351 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0924 13:48:10.241412 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:48:10.241536 (+ 124us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:48:10.241536 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:48:10.241538 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 13:48:10.241595 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:48:10.241596 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:48:10.242388 (+ 792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:48:10.242389 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:48:10.243739 (+ 1350us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:48:10.243744 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:48:10.244614 (+ 870us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:48:10.244616 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:48:10.244691 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:48:10.244696 (+ 5us) client_negotiation.cc:769] Sending connection context
0924 13:48:10.244758 (+ 62us) client_negotiation.cc:240] Negotiation successful
0924 13:48:10.244814 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"mutex_wait_us":42,"thread_start_us":60,"threads_started":1}
I20260924 13:48:10.245230 21820 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "47e5d842ec0042598a651c906036debc" candidate_uuid: "157ae8bbaa2c41c8b96615e2f6f5cecc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74fb0a58bbc04969a2fbe9c044f837ae" is_pre_election: true
I20260924 13:48:10.245332 21820 raft_consensus.cc:2454] T 47e5d842ec0042598a651c906036debc P 74fb0a58bbc04969a2fbe9c044f837ae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 157ae8bbaa2c41c8b96615e2f6f5cecc in term 0.
I20260924 13:48:10.245273 21680 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "47e5d842ec0042598a651c906036debc" candidate_uuid: "157ae8bbaa2c41c8b96615e2f6f5cecc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a66912dfe2404677a7a20ec0462d8ff0" is_pre_election: true
I20260924 13:48:10.245416 21680 raft_consensus.cc:2454] T 47e5d842ec0042598a651c906036debc P a66912dfe2404677a7a20ec0462d8ff0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 157ae8bbaa2c41c8b96615e2f6f5cecc in term 0.
I20260924 13:48:10.245532 21891 leader_election.cc:302] T 47e5d842ec0042598a651c906036debc P 157ae8bbaa2c41c8b96615e2f6f5cecc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 157ae8bbaa2c41c8b96615e2f6f5cecc, 74fb0a58bbc04969a2fbe9c044f837ae; no voters:
I20260924 13:48:10.245688 22041 raft_consensus.cc:2781] T 47e5d842ec0042598a651c906036debc P 157ae8bbaa2c41c8b96615e2f6f5cecc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 13:48:10.245723 22041 raft_consensus.cc:482] T 47e5d842ec0042598a651c906036debc P 157ae8bbaa2c41c8b96615e2f6f5cecc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 13:48:10.245741 22041 raft_consensus.cc:3037] T 47e5d842ec0042598a651c906036debc P 157ae8bbaa2c41c8b96615e2f6f5cecc [term 0 FOLLOWER]: Advancing to term 1
I20260924 13:48:10.246399 22041 raft_consensus.cc:504] T 47e5d842ec0042598a651c906036debc P 157ae8bbaa2c41c8b96615e2f6f5cecc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a66912dfe2404677a7a20ec0462d8ff0" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 45785 } } peers { permanent_uuid: "74fb0a58bbc04969a2fbe9c044f837ae" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 46159 } } peers { permanent_uuid: "157ae8bbaa2c41c8b96615e2f6f5cecc" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 46679 } }
I20260924 13:48:10.246537 22041 leader_election.cc:288] T 47e5d842ec0042598a651c906036debc P 157ae8bbaa2c41c8b96615e2f6f5cecc [CANDIDATE]: Term 1 election: Requested vote from peers a66912dfe2404677a7a20ec0462d8ff0 (127.13.149.193:45785), 74fb0a58bbc04969a2fbe9c044f837ae (127.13.149.194:46159)
I20260924 13:48:10.246726 21680 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "47e5d842ec0042598a651c906036debc" candidate_uuid: "157ae8bbaa2c41c8b96615e2f6f5cecc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a66912dfe2404677a7a20ec0462d8ff0"
I20260924 13:48:10.246763 21820 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "47e5d842ec0042598a651c906036debc" candidate_uuid: "157ae8bbaa2c41c8b96615e2f6f5cecc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74fb0a58bbc04969a2fbe9c044f837ae"
I20260924 13:48:10.246812 21680 raft_consensus.cc:3037] T 47e5d842ec0042598a651c906036debc P a66912dfe2404677a7a20ec0462d8ff0 [term 0 FOLLOWER]: Advancing to term 1
I20260924 13:48:10.246827 21820 raft_consensus.cc:3037] T 47e5d842ec0042598a651c906036debc P 74fb0a58bbc04969a2fbe9c044f837ae [term 0 FOLLOWER]: Advancing to term 1
I20260924 13:48:10.247395 21820 raft_consensus.cc:2454] T 47e5d842ec0042598a651c906036debc P 74fb0a58bbc04969a2fbe9c044f837ae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 157ae8bbaa2c41c8b96615e2f6f5cecc in term 1.
I20260924 13:48:10.247475 21680 raft_consensus.cc:2454] T 47e5d842ec0042598a651c906036debc P a66912dfe2404677a7a20ec0462d8ff0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 157ae8bbaa2c41c8b96615e2f6f5cecc in term 1.
I20260924 13:48:10.247572 21891 leader_election.cc:302] T 47e5d842ec0042598a651c906036debc P 157ae8bbaa2c41c8b96615e2f6f5cecc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 157ae8bbaa2c41c8b96615e2f6f5cecc, 74fb0a58bbc04969a2fbe9c044f837ae; no voters:
I20260924 13:48:10.247684 22041 raft_consensus.cc:2781] T 47e5d842ec0042598a651c906036debc P 157ae8bbaa2c41c8b96615e2f6f5cecc [term 1 FOLLOWER]: Leader election won for term 1
I20260924 13:48:10.247850 22041 raft_consensus.cc:686] T 47e5d842ec0042598a651c906036debc P 157ae8bbaa2c41c8b96615e2f6f5cecc [term 1 LEADER]: Becoming Leader. State: Replica: 157ae8bbaa2c41c8b96615e2f6f5cecc, State: Running, Role: LEADER
I20260924 13:48:10.247917 22041 consensus_queue.cc:237] T 47e5d842ec0042598a651c906036debc P 157ae8bbaa2c41c8b96615e2f6f5cecc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a66912dfe2404677a7a20ec0462d8ff0" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 45785 } } peers { permanent_uuid: "74fb0a58bbc04969a2fbe9c044f837ae" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 46159 } } peers { permanent_uuid: "157ae8bbaa2c41c8b96615e2f6f5cecc" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 46679 } }
I20260924 13:48:10.248654 21523 catalog_manager.cc:5495] T 47e5d842ec0042598a651c906036debc P 157ae8bbaa2c41c8b96615e2f6f5cecc reported cstate change: term changed from 0 to 1, leader changed from <none> to 157ae8bbaa2c41c8b96615e2f6f5cecc (127.13.149.195). New cstate: current_term: 1 leader_uuid: "157ae8bbaa2c41c8b96615e2f6f5cecc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a66912dfe2404677a7a20ec0462d8ff0" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 45785 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "74fb0a58bbc04969a2fbe9c044f837ae" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 46159 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "157ae8bbaa2c41c8b96615e2f6f5cecc" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 46679 } health_report { overall_health: HEALTHY } } }
Sep 24 13:48:10 dist-test-slave-sjl2 krb5kdc[21478](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257689, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.149.254@KRBTEST.COM for oryx/127.13.149.254@KRBTEST.COM
I20260924 13:48:10.309267 22071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:48:10.301740 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:42023 (local address 127.0.0.1:46478)
0924 13:48:10.301915 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0924 13:48:10.301925 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 13:48:10.301989 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:48:10.302204 (+ 215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:48:10.302205 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:48:10.302515 (+ 310us) client_negotiation.cc:189] Negotiated authn=SASL
0924 13:48:10.302566 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:48:10.302568 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:48:10.303239 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:48:10.303240 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:48:10.303758 (+ 518us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:48:10.303759 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:48:10.304120 (+ 361us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:48:10.304121 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:48:10.304165 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:48:10.304751 (+ 586us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 13:48:10.304756 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 13:48:10.306370 (+ 1614us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 13:48:10.308078 (+ 1708us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 13:48:10.308087 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 13:48:10.308090 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0924 13:48:10.308301 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 13:48:10.308570 (+ 269us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 13:48:10.308570 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 13:48:10.308571 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 13:48:10.308632 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 13:48:10.308965 (+ 333us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 13:48:10.308975 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 13:48:10.309025 (+ 50us) client_negotiation.cc:769] Sending connection context
0924 13:48:10.309085 (+ 60us) client_negotiation.cc:240] Negotiation successful
0924 13:48:10.309190 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":65,"threads_started":1}
I20260924 13:48:10.309329 21590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:48:10.301848 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46478 (local address 127.13.149.254:42023)
0924 13:48:10.301909 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0924 13:48:10.301911 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:48:10.302055 (+ 144us) server_negotiation.cc:407] Connection header received
0924 13:48:10.302095 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:48:10.302096 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:48:10.302110 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:48:10.302179 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:48:10.302652 (+ 473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:48:10.303173 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:48:10.303851 (+ 678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:48:10.304047 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:48:10.304113 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:48:10.306498 (+ 2385us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:48:10.306500 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:48:10.306500 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 13:48:10.306508 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:48:10.307974 (+ 1466us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:48:10.308423 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:48:10.308424 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:48:10.308424 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:48:10.308484 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:48:10.308721 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:48:10.308723 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:48:10.308723 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:48:10.308854 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:48:10.308946 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0924 13:48:10.309113 (+ 167us) server_negotiation.cc:299] Negotiation successful
0924 13:48:10.309241 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":56,"server-negotiator.queue_time_us":30}
I20260924 13:48:10.311491 21523 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.13.149.254@KRBTEST.COM'} at 127.0.0.1:46478:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260924 13:48:10.311702 21523 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 13:48:10.313481 21940 tablet_service.cc:1467] Processing CreateTablet for tablet 477bf2b1b0b1441e9b9940b4ef2c5962 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=aeadfd1c109445cabe11bff4fdcc4030]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260924 13:48:10.313489 21660 tablet_service.cc:1467] Processing CreateTablet for tablet 477bf2b1b0b1441e9b9940b4ef2c5962 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=aeadfd1c109445cabe11bff4fdcc4030]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260924 13:48:10.313640 21660 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 477bf2b1b0b1441e9b9940b4ef2c5962. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 13:48:10.313644 21940 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 477bf2b1b0b1441e9b9940b4ef2c5962. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 13:48:10.314996 21800 tablet_service.cc:1467] Processing CreateTablet for tablet 477bf2b1b0b1441e9b9940b4ef2c5962 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=aeadfd1c109445cabe11bff4fdcc4030]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260924 13:48:10.315152 21800 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 477bf2b1b0b1441e9b9940b4ef2c5962. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 13:48:10.316200 22034 tablet_bootstrap.cc:492] T 477bf2b1b0b1441e9b9940b4ef2c5962 P 74fb0a58bbc04969a2fbe9c044f837ae: Bootstrap starting.
I20260924 13:48:10.316212 22035 tablet_bootstrap.cc:492] T 477bf2b1b0b1441e9b9940b4ef2c5962 P 157ae8bbaa2c41c8b96615e2f6f5cecc: Bootstrap starting.
I20260924 13:48:10.316569 22034 tablet_bootstrap.cc:654] T 477bf2b1b0b1441e9b9940b4ef2c5962 P 74fb0a58bbc04969a2fbe9c044f837ae: Neither blocks nor log segments found. Creating new log.
I20260924 13:48:10.316587 22035 tablet_bootstrap.cc:654] T 477bf2b1b0b1441e9b9940b4ef2c5962 P 157ae8bbaa2c41c8b96615e2f6f5cecc: Neither blocks nor log segments found. Creating new log.
I20260924 13:48:10.316938 22037 tablet_bootstrap.cc:492] T 477bf2b1b0b1441e9b9940b4ef2c5962 P a66912dfe2404677a7a20ec0462d8ff0: Bootstrap starting.
I20260924 13:48:10.317256 22037 tablet_bootstrap.cc:654] T 477bf2b1b0b1441e9b9940b4ef2c5962 P a66912dfe2404677a7a20ec0462d8ff0: Neither blocks nor log segments found. Creating new log.
I20260924 13:48:10.317315 22034 tablet_bootstrap.cc:492] T 477bf2b1b0b1441e9b9940b4ef2c5962 P 74fb0a58bbc04969a2fbe9c044f837ae: No bootstrap required, opened a new log
I20260924 13:48:10.317368 22034 ts_tablet_manager.cc:1397] T 477bf2b1b0b1441e9b9940b4ef2c5962 P 74fb0a58bbc04969a2fbe9c044f837ae: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260924 13:48:10.317525 22034 raft_consensus.cc:348] T 477bf2b1b0b1441e9b9940b4ef2c5962 P 74fb0a58bbc04969a2fbe9c044f837ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a66912dfe2404677a7a20ec0462d8ff0" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 45785 } } peers { permanent_uuid: "74fb0a58bbc04969a2fbe9c044f837ae" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 46159 } } peers { permanent_uuid: "157ae8bbaa2c41c8b96615e2f6f5cecc" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 46679 } }
I20260924 13:48:10.317600 22034 raft_consensus.cc:374] T 477bf2b1b0b1441e9b9940b4ef2c5962 P 74fb0a58bbc04969a2fbe9c044f837ae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 13:48:10.317618 22034 raft_consensus.cc:729] T 477bf2b1b0b1441e9b9940b4ef2c5962 P 74fb0a58bbc04969a2fbe9c044f837ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 74fb0a58bbc04969a2fbe9c044f837ae, State: Initialized, Role: FOLLOWER
I20260924 13:48:10.317610 22035 tablet_bootstrap.cc:492] T 477bf2b1b0b1441e9b9940b4ef2c5962 P 157ae8bbaa2c41c8b96615e2f6f5cecc: No bootstrap required, opened a new log
I20260924 13:48:10.317663 22035 ts_tablet_manager.cc:1397] T 477bf2b1b0b1441e9b9940b4ef2c5962 P 157ae8bbaa2c41c8b96615e2f6f5cecc: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260924 13:48:10.317668 22034 consensus_queue.cc:260] T 477bf2b1b0b1441e9b9940b4ef2c5962 P 74fb0a58bbc04969a2fbe9c044f837ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a66912dfe2404677a7a20ec0462d8ff0" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 45785 } } peers { permanent_uuid: "74fb0a58bbc04969a2fbe9c044f837ae" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 46159 } } peers { permanent_uuid: "157ae8bbaa2c41c8b96615e2f6f5cecc" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 46679 } }
I20260924 13:48:10.317754 22034 ts_tablet_manager.cc:1428] T 477bf2b1b0b1441e9b9940b4ef2c5962 P 74fb0a58bbc04969a2fbe9c044f837ae: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260924 13:48:10.317812 22035 raft_consensus.cc:348] T 477bf2b1b0b1441e9b9940b4ef2c5962 P 157ae8bbaa2c41c8b96615e2f6f5cecc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a66912dfe2404677a7a20ec0462d8ff0" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 45785 } } peers { permanent_uuid: "74fb0a58bbc04969a2fbe9c044f837ae" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 46159 } } peers { permanent_uuid: "157ae8bbaa2c41c8b96615e2f6f5cecc" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 46679 } }
I20260924 13:48:10.317886 22035 raft_consensus.cc:374] T 477bf2b1b0b1441e9b9940b4ef2c5962 P 157ae8bbaa2c41c8b96615e2f6f5cecc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 13:48:10.317910 22035 raft_consensus.cc:729] T 477bf2b1b0b1441e9b9940b4ef2c5962 P 157ae8bbaa2c41c8b96615e2f6f5cecc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 157ae8bbaa2c41c8b96615e2f6f5cecc, State: Initialized, Role: FOLLOWER
I20260924 13:48:10.317888 22040 tablet_replica.cc:439] T 477bf2b1b0b1441e9b9940b4ef2c5962 P 74fb0a58bbc04969a2fbe9c044f837ae: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a66912dfe2404677a7a20ec0462d8ff0" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 45785 } } peers { permanent_uuid: "74fb0a58bbc04969a2fbe9c044f837ae" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 46159 } } peers { permanent_uuid: "157ae8bbaa2c41c8b96615e2f6f5cecc" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 46679 } } }
I20260924 13:48:10.317960 22040 tablet_replica.cc:442] T 477bf2b1b0b1441e9b9940b4ef2c5962 P 74fb0a58bbc04969a2fbe9c044f837ae: This TxnStatusTablet replica's current role is: FOLLOWER
I20260924 13:48:10.317952 22035 consensus_queue.cc:260] T 477bf2b1b0b1441e9b9940b4ef2c5962 P 157ae8bbaa2c41c8b96615e2f6f5cecc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a66912dfe2404677a7a20ec0462d8ff0" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 45785 } } peers { permanent_uuid: "74fb0a58bbc04969a2fbe9c044f837ae" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 46159 } } peers { permanent_uuid: "157ae8bbaa2c41c8b96615e2f6f5cecc" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 46679 } }
I20260924 13:48:10.318039 22035 ts_tablet_manager.cc:1428] T 477bf2b1b0b1441e9b9940b4ef2c5962 P 157ae8bbaa2c41c8b96615e2f6f5cecc: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260924 13:48:10.318197 22041 tablet_replica.cc:439] T 477bf2b1b0b1441e9b9940b4ef2c5962 P 157ae8bbaa2c41c8b96615e2f6f5cecc: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a66912dfe2404677a7a20ec0462d8ff0" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 45785 } } peers { permanent_uuid: "74fb0a58bbc04969a2fbe9c044f837ae" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 46159 } } peers { permanent_uuid: "157ae8bbaa2c41c8b96615e2f6f5cecc" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 46679 } } }
I20260924 13:48:10.318274 22041 tablet_replica.cc:442] T 477bf2b1b0b1441e9b9940b4ef2c5962 P 157ae8bbaa2c41c8b96615e2f6f5cecc: This TxnStatusTablet replica's current role is: FOLLOWER
I20260924 13:48:10.318565 22037 tablet_bootstrap.cc:492] T 477bf2b1b0b1441e9b9940b4ef2c5962 P a66912dfe2404677a7a20ec0462d8ff0: No bootstrap required, opened a new log
I20260924 13:48:10.318615 22037 ts_tablet_manager.cc:1397] T 477bf2b1b0b1441e9b9940b4ef2c5962 P a66912dfe2404677a7a20ec0462d8ff0: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260924 13:48:10.318730 22037 raft_consensus.cc:348] T 477bf2b1b0b1441e9b9940b4ef2c5962 P a66912dfe2404677a7a20ec0462d8ff0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a66912dfe2404677a7a20ec0462d8ff0" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 45785 } } peers { permanent_uuid: "74fb0a58bbc04969a2fbe9c044f837ae" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 46159 } } peers { permanent_uuid: "157ae8bbaa2c41c8b96615e2f6f5cecc" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 46679 } }
I20260924 13:48:10.318805 22037 raft_consensus.cc:374] T 477bf2b1b0b1441e9b9940b4ef2c5962 P a66912dfe2404677a7a20ec0462d8ff0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 13:48:10.318825 22037 raft_consensus.cc:729] T 477bf2b1b0b1441e9b9940b4ef2c5962 P a66912dfe2404677a7a20ec0462d8ff0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a66912dfe2404677a7a20ec0462d8ff0, State: Initialized, Role: FOLLOWER
I20260924 13:48:10.318861 22037 consensus_queue.cc:260] T 477bf2b1b0b1441e9b9940b4ef2c5962 P a66912dfe2404677a7a20ec0462d8ff0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a66912dfe2404677a7a20ec0462d8ff0" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 45785 } } peers { permanent_uuid: "74fb0a58bbc04969a2fbe9c044f837ae" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 46159 } } peers { permanent_uuid: "157ae8bbaa2c41c8b96615e2f6f5cecc" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 46679 } }
I20260924 13:48:10.318950 22037 ts_tablet_manager.cc:1428] T 477bf2b1b0b1441e9b9940b4ef2c5962 P a66912dfe2404677a7a20ec0462d8ff0: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260924 13:48:10.318989 22053 tablet_replica.cc:439] T 477bf2b1b0b1441e9b9940b4ef2c5962 P a66912dfe2404677a7a20ec0462d8ff0: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a66912dfe2404677a7a20ec0462d8ff0" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 45785 } } peers { permanent_uuid: "74fb0a58bbc04969a2fbe9c044f837ae" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 46159 } } peers { permanent_uuid: "157ae8bbaa2c41c8b96615e2f6f5cecc" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 46679 } } }
I20260924 13:48:10.319060 22053 tablet_replica.cc:442] T 477bf2b1b0b1441e9b9940b4ef2c5962 P a66912dfe2404677a7a20ec0462d8ff0: This TxnStatusTablet replica's current role is: FOLLOWER
I20260924 13:48:10.611996 22053 raft_consensus.cc:482] T 477bf2b1b0b1441e9b9940b4ef2c5962 P a66912dfe2404677a7a20ec0462d8ff0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 13:48:10.612119 22053 raft_consensus.cc:504] T 477bf2b1b0b1441e9b9940b4ef2c5962 P a66912dfe2404677a7a20ec0462d8ff0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a66912dfe2404677a7a20ec0462d8ff0" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 45785 } } peers { permanent_uuid: "74fb0a58bbc04969a2fbe9c044f837ae" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 46159 } } peers { permanent_uuid: "157ae8bbaa2c41c8b96615e2f6f5cecc" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 46679 } }
I20260924 13:48:10.612286 22053 leader_election.cc:288] T 477bf2b1b0b1441e9b9940b4ef2c5962 P a66912dfe2404677a7a20ec0462d8ff0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 74fb0a58bbc04969a2fbe9c044f837ae (127.13.149.194:46159), 157ae8bbaa2c41c8b96615e2f6f5cecc (127.13.149.195:46679)
I20260924 13:48:10.612550 21820 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "477bf2b1b0b1441e9b9940b4ef2c5962" candidate_uuid: "a66912dfe2404677a7a20ec0462d8ff0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74fb0a58bbc04969a2fbe9c044f837ae" is_pre_election: true
I20260924 13:48:10.612566 21960 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "477bf2b1b0b1441e9b9940b4ef2c5962" candidate_uuid: "a66912dfe2404677a7a20ec0462d8ff0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "157ae8bbaa2c41c8b96615e2f6f5cecc" is_pre_election: true
I20260924 13:48:10.612650 21820 raft_consensus.cc:2454] T 477bf2b1b0b1441e9b9940b4ef2c5962 P 74fb0a58bbc04969a2fbe9c044f837ae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a66912dfe2404677a7a20ec0462d8ff0 in term 0.
I20260924 13:48:10.612650 21960 raft_consensus.cc:2454] T 477bf2b1b0b1441e9b9940b4ef2c5962 P 157ae8bbaa2c41c8b96615e2f6f5cecc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a66912dfe2404677a7a20ec0462d8ff0 in term 0.
I20260924 13:48:10.612854 21611 leader_election.cc:302] T 477bf2b1b0b1441e9b9940b4ef2c5962 P a66912dfe2404677a7a20ec0462d8ff0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74fb0a58bbc04969a2fbe9c044f837ae, a66912dfe2404677a7a20ec0462d8ff0; no voters:
I20260924 13:48:10.612991 22053 raft_consensus.cc:2781] T 477bf2b1b0b1441e9b9940b4ef2c5962 P a66912dfe2404677a7a20ec0462d8ff0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 13:48:10.613042 22053 raft_consensus.cc:482] T 477bf2b1b0b1441e9b9940b4ef2c5962 P a66912dfe2404677a7a20ec0462d8ff0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 13:48:10.613067 22053 raft_consensus.cc:3037] T 477bf2b1b0b1441e9b9940b4ef2c5962 P a66912dfe2404677a7a20ec0462d8ff0 [term 0 FOLLOWER]: Advancing to term 1
I20260924 13:48:10.613957 22053 raft_consensus.cc:504] T 477bf2b1b0b1441e9b9940b4ef2c5962 P a66912dfe2404677a7a20ec0462d8ff0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a66912dfe2404677a7a20ec0462d8ff0" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 45785 } } peers { permanent_uuid: "74fb0a58bbc04969a2fbe9c044f837ae" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 46159 } } peers { permanent_uuid: "157ae8bbaa2c41c8b96615e2f6f5cecc" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 46679 } }
I20260924 13:48:10.614106 22053 leader_election.cc:288] T 477bf2b1b0b1441e9b9940b4ef2c5962 P a66912dfe2404677a7a20ec0462d8ff0 [CANDIDATE]: Term 1 election: Requested vote from peers 74fb0a58bbc04969a2fbe9c044f837ae (127.13.149.194:46159), 157ae8bbaa2c41c8b96615e2f6f5cecc (127.13.149.195:46679)
I20260924 13:48:10.614305 21820 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "477bf2b1b0b1441e9b9940b4ef2c5962" candidate_uuid: "a66912dfe2404677a7a20ec0462d8ff0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74fb0a58bbc04969a2fbe9c044f837ae"
I20260924 13:48:10.614329 21960 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "477bf2b1b0b1441e9b9940b4ef2c5962" candidate_uuid: "a66912dfe2404677a7a20ec0462d8ff0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "157ae8bbaa2c41c8b96615e2f6f5cecc"
I20260924 13:48:10.614385 21820 raft_consensus.cc:3037] T 477bf2b1b0b1441e9b9940b4ef2c5962 P 74fb0a58bbc04969a2fbe9c044f837ae [term 0 FOLLOWER]: Advancing to term 1
I20260924 13:48:10.614408 21960 raft_consensus.cc:3037] T 477bf2b1b0b1441e9b9940b4ef2c5962 P 157ae8bbaa2c41c8b96615e2f6f5cecc [term 0 FOLLOWER]: Advancing to term 1
I20260924 13:48:10.615077 21960 raft_consensus.cc:2454] T 477bf2b1b0b1441e9b9940b4ef2c5962 P 157ae8bbaa2c41c8b96615e2f6f5cecc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a66912dfe2404677a7a20ec0462d8ff0 in term 1.
I20260924 13:48:10.615089 21820 raft_consensus.cc:2454] T 477bf2b1b0b1441e9b9940b4ef2c5962 P 74fb0a58bbc04969a2fbe9c044f837ae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a66912dfe2404677a7a20ec0462d8ff0 in term 1.
I20260924 13:48:10.615289 21611 leader_election.cc:302] T 477bf2b1b0b1441e9b9940b4ef2c5962 P a66912dfe2404677a7a20ec0462d8ff0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74fb0a58bbc04969a2fbe9c044f837ae, a66912dfe2404677a7a20ec0462d8ff0; no voters:
I20260924 13:48:10.615408 22053 raft_consensus.cc:2781] T 477bf2b1b0b1441e9b9940b4ef2c5962 P a66912dfe2404677a7a20ec0462d8ff0 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 13:48:10.615532 22053 raft_consensus.cc:686] T 477bf2b1b0b1441e9b9940b4ef2c5962 P a66912dfe2404677a7a20ec0462d8ff0 [term 1 LEADER]: Becoming Leader. State: Replica: a66912dfe2404677a7a20ec0462d8ff0, State: Running, Role: LEADER
I20260924 13:48:10.615583 22053 consensus_queue.cc:237] T 477bf2b1b0b1441e9b9940b4ef2c5962 P a66912dfe2404677a7a20ec0462d8ff0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a66912dfe2404677a7a20ec0462d8ff0" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 45785 } } peers { permanent_uuid: "74fb0a58bbc04969a2fbe9c044f837ae" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 46159 } } peers { permanent_uuid: "157ae8bbaa2c41c8b96615e2f6f5cecc" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 46679 } }
I20260924 13:48:10.616050 22075 tablet_replica.cc:439] T 477bf2b1b0b1441e9b9940b4ef2c5962 P a66912dfe2404677a7a20ec0462d8ff0: TxnStatusTablet state changed. Reason: New leader a66912dfe2404677a7a20ec0462d8ff0. Latest consensus state: current_term: 1 leader_uuid: "a66912dfe2404677a7a20ec0462d8ff0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a66912dfe2404677a7a20ec0462d8ff0" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 45785 } } peers { permanent_uuid: "74fb0a58bbc04969a2fbe9c044f837ae" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 46159 } } peers { permanent_uuid: "157ae8bbaa2c41c8b96615e2f6f5cecc" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 46679 } } }
I20260924 13:48:10.616158 22075 tablet_replica.cc:442] T 477bf2b1b0b1441e9b9940b4ef2c5962 P a66912dfe2404677a7a20ec0462d8ff0: This TxnStatusTablet replica's current role is: LEADER
I20260924 13:48:10.616348 22077 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260924 13:48:10.616492 22077 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260924 13:48:10.616405 21522 catalog_manager.cc:5495] T 477bf2b1b0b1441e9b9940b4ef2c5962 P a66912dfe2404677a7a20ec0462d8ff0 reported cstate change: term changed from 0 to 1, leader changed from <none> to a66912dfe2404677a7a20ec0462d8ff0 (127.13.149.193). New cstate: current_term: 1 leader_uuid: "a66912dfe2404677a7a20ec0462d8ff0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a66912dfe2404677a7a20ec0462d8ff0" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 45785 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "74fb0a58bbc04969a2fbe9c044f837ae" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 46159 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "157ae8bbaa2c41c8b96615e2f6f5cecc" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 46679 } health_report { overall_health: UNKNOWN } } }
I20260924 13:48:10.672552 22030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:48:10.669977 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34906 (local address 127.13.149.193:45785)
0924 13:48:10.670029 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0924 13:48:10.670031 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:48:10.670041 (+ 10us) server_negotiation.cc:407] Connection header received
0924 13:48:10.670123 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:48:10.670124 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:48:10.670131 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:48:10.670179 (+ 48us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 13:48:10.670308 (+ 129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:48:10.670838 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:48:10.671923 (+ 1085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:48:10.672116 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:48:10.672183 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:48:10.672349 (+ 166us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 13:48:10.672377 (+ 28us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 13:48:10.672408 (+ 31us) server_negotiation.cc:1035] Waiting for connection context
0924 13:48:10.672465 (+ 57us) server_negotiation.cc:299] Negotiation successful
0924 13:48:10.672484 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260924 13:48:10.672597 22071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:48:10.669923 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.193:45785 (local address 127.0.0.1:34906)
0924 13:48:10.670004 (+ 81us) negotiation.cc:107] Waiting for socket to connect
0924 13:48:10.670015 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0924 13:48:10.670056 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:48:10.670196 (+ 140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:48:10.670197 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:48:10.670198 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0924 13:48:10.670254 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:48:10.670256 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:48:10.670909 (+ 653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:48:10.670910 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:48:10.671822 (+ 912us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:48:10.671825 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:48:10.672188 (+ 363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:48:10.672189 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:48:10.672252 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:48:10.672274 (+ 22us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0924 13:48:10.672420 (+ 146us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0924 13:48:10.672426 (+ 6us) client_negotiation.cc:769] Sending connection context
0924 13:48:10.672462 (+ 36us) client_negotiation.cc:240] Negotiation successful
0924 13:48:10.672514 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":48}
I20260924 13:48:10.673816 21820 raft_consensus.cc:1261] T 477bf2b1b0b1441e9b9940b4ef2c5962 P 74fb0a58bbc04969a2fbe9c044f837ae [term 1 FOLLOWER]: Refusing update from remote peer a66912dfe2404677a7a20ec0462d8ff0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 13:48:10.673816 21960 raft_consensus.cc:1261] T 477bf2b1b0b1441e9b9940b4ef2c5962 P 157ae8bbaa2c41c8b96615e2f6f5cecc [term 1 FOLLOWER]: Refusing update from remote peer a66912dfe2404677a7a20ec0462d8ff0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 13:48:10.674044 22053 consensus_queue.cc:1035] T 477bf2b1b0b1441e9b9940b4ef2c5962 P a66912dfe2404677a7a20ec0462d8ff0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "74fb0a58bbc04969a2fbe9c044f837ae" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 46159 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 13:48:10.674187 22075 consensus_queue.cc:1035] T 477bf2b1b0b1441e9b9940b4ef2c5962 P a66912dfe2404677a7a20ec0462d8ff0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "157ae8bbaa2c41c8b96615e2f6f5cecc" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 46679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 13:48:10.674654 22040 tablet_replica.cc:439] T 477bf2b1b0b1441e9b9940b4ef2c5962 P 74fb0a58bbc04969a2fbe9c044f837ae: TxnStatusTablet state changed. Reason: New leader a66912dfe2404677a7a20ec0462d8ff0. Latest consensus state: current_term: 1 leader_uuid: "a66912dfe2404677a7a20ec0462d8ff0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a66912dfe2404677a7a20ec0462d8ff0" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 45785 } } peers { permanent_uuid: "74fb0a58bbc04969a2fbe9c044f837ae" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 46159 } } peers { permanent_uuid: "157ae8bbaa2c41c8b96615e2f6f5cecc" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 46679 } } }
I20260924 13:48:10.674754 22040 tablet_replica.cc:442] T 477bf2b1b0b1441e9b9940b4ef2c5962 P 74fb0a58bbc04969a2fbe9c044f837ae: This TxnStatusTablet replica's current role is: FOLLOWER
I20260924 13:48:10.674736 22041 tablet_replica.cc:439] T 477bf2b1b0b1441e9b9940b4ef2c5962 P 157ae8bbaa2c41c8b96615e2f6f5cecc: TxnStatusTablet state changed. Reason: New leader a66912dfe2404677a7a20ec0462d8ff0. Latest consensus state: current_term: 1 leader_uuid: "a66912dfe2404677a7a20ec0462d8ff0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a66912dfe2404677a7a20ec0462d8ff0" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 45785 } } peers { permanent_uuid: "74fb0a58bbc04969a2fbe9c044f837ae" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 46159 } } peers { permanent_uuid: "157ae8bbaa2c41c8b96615e2f6f5cecc" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 46679 } } }
I20260924 13:48:10.674820 22041 tablet_replica.cc:442] T 477bf2b1b0b1441e9b9940b4ef2c5962 P 157ae8bbaa2c41c8b96615e2f6f5cecc: This TxnStatusTablet replica's current role is: FOLLOWER
I20260924 13:48:10.675781 22053 tablet_replica.cc:439] T 477bf2b1b0b1441e9b9940b4ef2c5962 P a66912dfe2404677a7a20ec0462d8ff0: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "a66912dfe2404677a7a20ec0462d8ff0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a66912dfe2404677a7a20ec0462d8ff0" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 45785 } } peers { permanent_uuid: "74fb0a58bbc04969a2fbe9c044f837ae" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 46159 } } peers { permanent_uuid: "157ae8bbaa2c41c8b96615e2f6f5cecc" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 46679 } } }
I20260924 13:48:10.675868 22053 tablet_replica.cc:442] T 477bf2b1b0b1441e9b9940b4ef2c5962 P a66912dfe2404677a7a20ec0462d8ff0: This TxnStatusTablet replica's current role is: LEADER
I20260924 13:48:10.675912 22053 tablet_replica.cc:439] T 477bf2b1b0b1441e9b9940b4ef2c5962 P a66912dfe2404677a7a20ec0462d8ff0: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "a66912dfe2404677a7a20ec0462d8ff0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a66912dfe2404677a7a20ec0462d8ff0" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 45785 } } peers { permanent_uuid: "74fb0a58bbc04969a2fbe9c044f837ae" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 46159 } } peers { permanent_uuid: "157ae8bbaa2c41c8b96615e2f6f5cecc" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 46679 } } }
I20260924 13:48:10.675951 22053 tablet_replica.cc:442] T 477bf2b1b0b1441e9b9940b4ef2c5962 P a66912dfe2404677a7a20ec0462d8ff0: This TxnStatusTablet replica's current role is: LEADER
I20260924 13:48:10.675993 22080 mvcc.cc:205] Tried to move back new op lower bound from 7332895500998270976 to 7332895500761772032. Current Snapshot: MvccSnapshot[applied={T|T < 7332895500998270976 or (T in {7332895500998270976})}]
I20260924 13:48:10.676177 22041 tablet_replica.cc:439] T 477bf2b1b0b1441e9b9940b4ef2c5962 P 157ae8bbaa2c41c8b96615e2f6f5cecc: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "a66912dfe2404677a7a20ec0462d8ff0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a66912dfe2404677a7a20ec0462d8ff0" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 45785 } } peers { permanent_uuid: "74fb0a58bbc04969a2fbe9c044f837ae" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 46159 } } peers { permanent_uuid: "157ae8bbaa2c41c8b96615e2f6f5cecc" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 46679 } } }
I20260924 13:48:10.676261 22041 tablet_replica.cc:442] T 477bf2b1b0b1441e9b9940b4ef2c5962 P 157ae8bbaa2c41c8b96615e2f6f5cecc: This TxnStatusTablet replica's current role is: FOLLOWER
I20260924 13:48:10.676368 22083 mvcc.cc:205] Tried to move back new op lower bound from 7332895500998270976 to 7332895500761772032. Current Snapshot: MvccSnapshot[applied={T|T < 7332895500998270976}]
I20260924 13:48:10.676645 22081 mvcc.cc:205] Tried to move back new op lower bound from 7332895500998270976 to 7332895500761772032. Current Snapshot: MvccSnapshot[applied={T|T < 7332895500998270976}]
I20260924 13:48:10.676796 22040 tablet_replica.cc:439] T 477bf2b1b0b1441e9b9940b4ef2c5962 P 74fb0a58bbc04969a2fbe9c044f837ae: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "a66912dfe2404677a7a20ec0462d8ff0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a66912dfe2404677a7a20ec0462d8ff0" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 45785 } } peers { permanent_uuid: "74fb0a58bbc04969a2fbe9c044f837ae" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 46159 } } peers { permanent_uuid: "157ae8bbaa2c41c8b96615e2f6f5cecc" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 46679 } } }
I20260924 13:48:10.676877 22040 tablet_replica.cc:442] T 477bf2b1b0b1441e9b9940b4ef2c5962 P 74fb0a58bbc04969a2fbe9c044f837ae: This TxnStatusTablet replica's current role is: FOLLOWER
I20260924 13:48:10.679927 22041 consensus_queue.cc:1035] T 47e5d842ec0042598a651c906036debc P 157ae8bbaa2c41c8b96615e2f6f5cecc [LEADER]: Connected to new peer: Peer: permanent_uuid: "74fb0a58bbc04969a2fbe9c044f837ae" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 46159 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 13:48:10.681294 22091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:48:10.678103 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42756 (local address 127.13.149.195:46679)
0924 13:48:10.678207 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0924 13:48:10.678209 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:48:10.678237 (+ 28us) server_negotiation.cc:407] Connection header received
0924 13:48:10.678282 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:48:10.678283 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:48:10.678304 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:48:10.678371 (+ 67us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 13:48:10.678478 (+ 107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:48:10.679239 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:48:10.680186 (+ 947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:48:10.680474 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:48:10.680539 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:48:10.680745 (+ 206us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 13:48:10.680879 (+ 134us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 13:48:10.680967 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0924 13:48:10.681189 (+ 222us) server_negotiation.cc:299] Negotiation successful
0924 13:48:10.681213 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84,"thread_start_us":61,"threads_started":1}
I20260924 13:48:10.681582 22064 consensus_queue.cc:1035] T 47e5d842ec0042598a651c906036debc P 157ae8bbaa2c41c8b96615e2f6f5cecc [LEADER]: Connected to new peer: Peer: permanent_uuid: "a66912dfe2404677a7a20ec0462d8ff0" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 45785 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 13:48:10.686842 22096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:48:10.684263 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.193:45785 (local address 127.13.149.195:38115)
0924 13:48:10.684398 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0924 13:48:10.684408 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 13:48:10.684503 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:48:10.684662 (+ 159us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:48:10.684663 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:48:10.684664 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0924 13:48:10.684723 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:48:10.684742 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:48:10.685406 (+ 664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:48:10.685407 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:48:10.686084 (+ 677us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:48:10.686088 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:48:10.686430 (+ 342us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:48:10.686431 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:48:10.686494 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:48:10.686512 (+ 18us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0924 13:48:10.686670 (+ 158us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0924 13:48:10.686671 (+ 1us) client_negotiation.cc:769] Sending connection context
0924 13:48:10.686707 (+ 36us) client_negotiation.cc:240] Negotiation successful
0924 13:48:10.686759 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":96,"thread_start_us":70,"threads_started":1}
I20260924 13:48:10.686827 22030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:48:10.684335 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.149.195:38115 (local address 127.13.149.193:45785)
0924 13:48:10.684400 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0924 13:48:10.684401 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0924 13:48:10.684499 (+ 98us) server_negotiation.cc:407] Connection header received
0924 13:48:10.684551 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:48:10.684552 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:48:10.684563 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:48:10.684621 (+ 58us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 13:48:10.684785 (+ 164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:48:10.685332 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:48:10.686161 (+ 829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:48:10.686365 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:48:10.686421 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:48:10.686581 (+ 160us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 13:48:10.686618 (+ 37us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 13:48:10.686660 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0924 13:48:10.686719 (+ 59us) server_negotiation.cc:299] Negotiation successful
0924 13:48:10.686742 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260924 13:48:10.698300 22091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:48:10.695561 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.149.193:59015 (local address 127.13.149.195:46679)
0924 13:48:10.695624 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0924 13:48:10.695626 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:48:10.695634 (+ 8us) server_negotiation.cc:407] Connection header received
0924 13:48:10.695705 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:48:10.695705 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:48:10.695715 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:48:10.695760 (+ 45us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 13:48:10.695898 (+ 138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:48:10.696382 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:48:10.697565 (+ 1183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:48:10.697816 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:48:10.697868 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:48:10.698074 (+ 206us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 13:48:10.698101 (+ 27us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 13:48:10.698132 (+ 31us) server_negotiation.cc:1035] Waiting for connection context
0924 13:48:10.698201 (+ 69us) server_negotiation.cc:299] Negotiation successful
0924 13:48:10.698220 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260924 13:48:10.698483 22099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:48:10.695359 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.195:46679 (local address 127.13.149.193:59015)
0924 13:48:10.695557 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0924 13:48:10.695571 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0924 13:48:10.695647 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:48:10.695765 (+ 118us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:48:10.695766 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:48:10.695768 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0924 13:48:10.695826 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:48:10.695828 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:48:10.696461 (+ 633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:48:10.696462 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:48:10.697409 (+ 947us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:48:10.697412 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:48:10.697901 (+ 489us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:48:10.697902 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:48:10.697967 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:48:10.697986 (+ 19us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0924 13:48:10.698154 (+ 168us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0924 13:48:10.698156 (+ 2us) client_negotiation.cc:769] Sending connection context
0924 13:48:10.698202 (+ 46us) client_negotiation.cc:240] Negotiation successful
0924 13:48:10.698273 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":93,"threads_started":1}
I20260924 13:48:10.709975 21523 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46476:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 13:48:10.710109 21523 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46476:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 13:48:10.710848 21523 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9f6c1d2f544b490eb7522be8ff50ceba: Sending DeleteTablet for 3 replicas of tablet 47e5d842ec0042598a651c906036debc
I20260924 13:48:10.711177 21940 tablet_service.cc:1514] Processing DeleteTablet for tablet 47e5d842ec0042598a651c906036debc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 13:48:10 UTC) from {username='oryx', principal='oryx/127.13.149.254@KRBTEST.COM'} at 127.0.0.1:42746
I20260924 13:48:10.711467 22102 tablet_replica.cc:330] T 47e5d842ec0042598a651c906036debc P 157ae8bbaa2c41c8b96615e2f6f5cecc: stopping tablet replica
I20260924 13:48:10.711544 22102 raft_consensus.cc:2229] T 47e5d842ec0042598a651c906036debc P 157ae8bbaa2c41c8b96615e2f6f5cecc [term 1 LEADER]: Raft consensus shutting down.
I20260924 13:48:10.711659 22102 raft_consensus.cc:2258] T 47e5d842ec0042598a651c906036debc P 157ae8bbaa2c41c8b96615e2f6f5cecc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 13:48:10.711859 21800 tablet_service.cc:1514] Processing DeleteTablet for tablet 47e5d842ec0042598a651c906036debc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 13:48:10 UTC) from {username='oryx', principal='oryx/127.13.149.254@KRBTEST.COM'} at 127.0.0.1:47830
I20260924 13:48:10.711975 22102 ts_tablet_manager.cc:1905] T 47e5d842ec0042598a651c906036debc P 157ae8bbaa2c41c8b96615e2f6f5cecc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 13:48:10.712180 21660 tablet_service.cc:1514] Processing DeleteTablet for tablet 47e5d842ec0042598a651c906036debc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 13:48:10 UTC) from {username='oryx', principal='oryx/127.13.149.254@KRBTEST.COM'} at 127.0.0.1:34888
I20260924 13:48:10.713945 22102 ts_tablet_manager.cc:1918] T 47e5d842ec0042598a651c906036debc P 157ae8bbaa2c41c8b96615e2f6f5cecc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260924 13:48:10.714010 22102 log.cc:1196] T 47e5d842ec0042598a651c906036debc P 157ae8bbaa2c41c8b96615e2f6f5cecc: Deleting WAL directory at /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790257629041645-13911-0/minicluster-data/ts-2/wal/wals/47e5d842ec0042598a651c906036debc
I20260924 13:48:10.714216 22102 ts_tablet_manager.cc:1939] T 47e5d842ec0042598a651c906036debc P 157ae8bbaa2c41c8b96615e2f6f5cecc: Deleting consensus metadata
I20260924 13:48:10.714635 21507 catalog_manager.cc:4841] TS 157ae8bbaa2c41c8b96615e2f6f5cecc (127.13.149.195:46679): tablet 47e5d842ec0042598a651c906036debc (table test-table [id=e78a1ae8505147b2bd6f123195763ec8]) successfully deleted
I20260924 13:48:10.718703 22103 tablet_replica.cc:330] T 47e5d842ec0042598a651c906036debc P 74fb0a58bbc04969a2fbe9c044f837ae: stopping tablet replica
I20260924 13:48:10.718933 22104 tablet_replica.cc:330] T 47e5d842ec0042598a651c906036debc P a66912dfe2404677a7a20ec0462d8ff0: stopping tablet replica
I20260924 13:48:10.719082 22104 raft_consensus.cc:2229] T 47e5d842ec0042598a651c906036debc P a66912dfe2404677a7a20ec0462d8ff0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 13:48:10.719182 22103 raft_consensus.cc:2229] T 47e5d842ec0042598a651c906036debc P 74fb0a58bbc04969a2fbe9c044f837ae [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 13:48:10.719295 22104 raft_consensus.cc:2258] T 47e5d842ec0042598a651c906036debc P a66912dfe2404677a7a20ec0462d8ff0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 13:48:10.719370 22103 raft_consensus.cc:2258] T 47e5d842ec0042598a651c906036debc P 74fb0a58bbc04969a2fbe9c044f837ae [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 13:48:10.719776 22104 ts_tablet_manager.cc:1905] T 47e5d842ec0042598a651c906036debc P a66912dfe2404677a7a20ec0462d8ff0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 13:48:10.720101 22103 ts_tablet_manager.cc:1905] T 47e5d842ec0042598a651c906036debc P 74fb0a58bbc04969a2fbe9c044f837ae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 13:48:10.722021 22104 ts_tablet_manager.cc:1918] T 47e5d842ec0042598a651c906036debc P a66912dfe2404677a7a20ec0462d8ff0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260924 13:48:10.722165 22104 log.cc:1196] T 47e5d842ec0042598a651c906036debc P a66912dfe2404677a7a20ec0462d8ff0: Deleting WAL directory at /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790257629041645-13911-0/minicluster-data/ts-0/wal/wals/47e5d842ec0042598a651c906036debc
I20260924 13:48:10.722472 22104 ts_tablet_manager.cc:1939] T 47e5d842ec0042598a651c906036debc P a66912dfe2404677a7a20ec0462d8ff0: Deleting consensus metadata
I20260924 13:48:10.722829 22103 ts_tablet_manager.cc:1918] T 47e5d842ec0042598a651c906036debc P 74fb0a58bbc04969a2fbe9c044f837ae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260924 13:48:10.722916 22103 log.cc:1196] T 47e5d842ec0042598a651c906036debc P 74fb0a58bbc04969a2fbe9c044f837ae: Deleting WAL directory at /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790257629041645-13911-0/minicluster-data/ts-1/wal/wals/47e5d842ec0042598a651c906036debc
I20260924 13:48:10.722934 21507 catalog_manager.cc:4841] TS a66912dfe2404677a7a20ec0462d8ff0 (127.13.149.193:45785): tablet 47e5d842ec0042598a651c906036debc (table test-table [id=e78a1ae8505147b2bd6f123195763ec8]) successfully deleted
I20260924 13:48:10.724715 22103 ts_tablet_manager.cc:1939] T 47e5d842ec0042598a651c906036debc P 74fb0a58bbc04969a2fbe9c044f837ae: Deleting consensus metadata
I20260924 13:48:10.725169 21507 catalog_manager.cc:4841] TS 74fb0a58bbc04969a2fbe9c044f837ae (127.13.149.194:46159): tablet 47e5d842ec0042598a651c906036debc (table test-table [id=e78a1ae8505147b2bd6f123195763ec8]) successfully deleted
Sep 24 13:48:10 dist-test-slave-sjl2 krb5kdc[21478](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257690, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 24 13:48:10 dist-test-slave-sjl2 krb5kdc[21478](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.13.149.254@KRBTEST.COM, Server not found in Kerberos database
Sep 24 13:48:10 dist-test-slave-sjl2 krb5kdc[21478](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.13.149.254@KRBTEST.COM, Server not found in Kerberos database
W20260924 13:48:10.748329 21590 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 13:48:10.741228 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46488 (local address 127.13.149.254:42023)
0924 13:48:10.741277 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0924 13:48:10.741279 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:48:10.741438 (+ 159us) server_negotiation.cc:407] Connection header received
0924 13:48:10.741558 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:48:10.741559 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:48:10.741573 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:48:10.741636 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:48:10.742350 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:48:10.743132 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:48:10.744252 (+ 1120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:48:10.744541 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:48:10.744612 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:48:10.748254 (+ 3642us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46488: BlockingRecv error: recv got EOF from 127.0.0.1:46488 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
Sep 24 13:48:10 dist-test-slave-sjl2 krb5kdc[21478](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257690, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.13.149.254@KRBTEST.COM
I20260924 13:48:10.758239 21590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:48:10.749422 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46490 (local address 127.13.149.254:42023)
0924 13:48:10.749473 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0924 13:48:10.749475 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:48:10.749520 (+ 45us) server_negotiation.cc:407] Connection header received
0924 13:48:10.749587 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:48:10.749588 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:48:10.749598 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:48:10.749665 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:48:10.750218 (+ 553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:48:10.751002 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:48:10.752033 (+ 1031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:48:10.752341 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:48:10.752414 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:48:10.755211 (+ 2797us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:48:10.755214 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:48:10.755214 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 13:48:10.755224 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:48:10.756866 (+ 1642us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:48:10.757351 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:48:10.757352 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:48:10.757352 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:48:10.757416 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:48:10.757673 (+ 257us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:48:10.757674 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:48:10.757674 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:48:10.757863 (+ 189us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:48:10.757952 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0924 13:48:10.758086 (+ 134us) server_negotiation.cc:299] Negotiation successful
0924 13:48:10.758166 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260924 13:48:10.760450 21523 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46490:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 13:48:10.760887 21523 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 13:48:10.762441 21800 tablet_service.cc:1467] Processing CreateTablet for tablet 647f95b6f1f24fddbc52d68590da28c3 (DEFAULT_TABLE table=test-table [id=76844b46e06642e192a32dc5e14ec6db]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 13:48:10.762470 21940 tablet_service.cc:1467] Processing CreateTablet for tablet 647f95b6f1f24fddbc52d68590da28c3 (DEFAULT_TABLE table=test-table [id=76844b46e06642e192a32dc5e14ec6db]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 13:48:10.762619 21800 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 647f95b6f1f24fddbc52d68590da28c3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 13:48:10.762703 21940 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 647f95b6f1f24fddbc52d68590da28c3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 13:48:10.764549 21660 tablet_service.cc:1467] Processing CreateTablet for tablet 647f95b6f1f24fddbc52d68590da28c3 (DEFAULT_TABLE table=test-table [id=76844b46e06642e192a32dc5e14ec6db]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 13:48:10.764698 21660 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 647f95b6f1f24fddbc52d68590da28c3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 13:48:10.766065 22037 tablet_bootstrap.cc:492] T 647f95b6f1f24fddbc52d68590da28c3 P a66912dfe2404677a7a20ec0462d8ff0: Bootstrap starting.
I20260924 13:48:10.766093 22034 tablet_bootstrap.cc:492] T 647f95b6f1f24fddbc52d68590da28c3 P 74fb0a58bbc04969a2fbe9c044f837ae: Bootstrap starting.
I20260924 13:48:10.766410 22037 tablet_bootstrap.cc:654] T 647f95b6f1f24fddbc52d68590da28c3 P a66912dfe2404677a7a20ec0462d8ff0: Neither blocks nor log segments found. Creating new log.
I20260924 13:48:10.766501 22034 tablet_bootstrap.cc:654] T 647f95b6f1f24fddbc52d68590da28c3 P 74fb0a58bbc04969a2fbe9c044f837ae: Neither blocks nor log segments found. Creating new log.
I20260924 13:48:10.766840 22035 tablet_bootstrap.cc:492] T 647f95b6f1f24fddbc52d68590da28c3 P 157ae8bbaa2c41c8b96615e2f6f5cecc: Bootstrap starting.
I20260924 13:48:10.766927 22037 tablet_bootstrap.cc:492] T 647f95b6f1f24fddbc52d68590da28c3 P a66912dfe2404677a7a20ec0462d8ff0: No bootstrap required, opened a new log
I20260924 13:48:10.766973 22037 ts_tablet_manager.cc:1397] T 647f95b6f1f24fddbc52d68590da28c3 P a66912dfe2404677a7a20ec0462d8ff0: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260924 13:48:10.767079 22037 raft_consensus.cc:348] T 647f95b6f1f24fddbc52d68590da28c3 P a66912dfe2404677a7a20ec0462d8ff0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a66912dfe2404677a7a20ec0462d8ff0" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 45785 } } peers { permanent_uuid: "74fb0a58bbc04969a2fbe9c044f837ae" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 46159 } } peers { permanent_uuid: "157ae8bbaa2c41c8b96615e2f6f5cecc" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 46679 } }
I20260924 13:48:10.767145 22037 raft_consensus.cc:374] T 647f95b6f1f24fddbc52d68590da28c3 P a66912dfe2404677a7a20ec0462d8ff0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 13:48:10.767163 22037 raft_consensus.cc:729] T 647f95b6f1f24fddbc52d68590da28c3 P a66912dfe2404677a7a20ec0462d8ff0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a66912dfe2404677a7a20ec0462d8ff0, State: Initialized, Role: FOLLOWER
I20260924 13:48:10.767211 22037 consensus_queue.cc:260] T 647f95b6f1f24fddbc52d68590da28c3 P a66912dfe2404677a7a20ec0462d8ff0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a66912dfe2404677a7a20ec0462d8ff0" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 45785 } } peers { permanent_uuid: "74fb0a58bbc04969a2fbe9c044f837ae" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 46159 } } peers { permanent_uuid: "157ae8bbaa2c41c8b96615e2f6f5cecc" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 46679 } }
I20260924 13:48:10.767293 22037 ts_tablet_manager.cc:1428] T 647f95b6f1f24fddbc52d68590da28c3 P a66912dfe2404677a7a20ec0462d8ff0: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260924 13:48:10.767297 22035 tablet_bootstrap.cc:654] T 647f95b6f1f24fddbc52d68590da28c3 P 157ae8bbaa2c41c8b96615e2f6f5cecc: Neither blocks nor log segments found. Creating new log.
I20260924 13:48:10.768225 22035 tablet_bootstrap.cc:492] T 647f95b6f1f24fddbc52d68590da28c3 P 157ae8bbaa2c41c8b96615e2f6f5cecc: No bootstrap required, opened a new log
I20260924 13:48:10.768285 22035 ts_tablet_manager.cc:1397] T 647f95b6f1f24fddbc52d68590da28c3 P 157ae8bbaa2c41c8b96615e2f6f5cecc: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260924 13:48:10.768424 22035 raft_consensus.cc:348] T 647f95b6f1f24fddbc52d68590da28c3 P 157ae8bbaa2c41c8b96615e2f6f5cecc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a66912dfe2404677a7a20ec0462d8ff0" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 45785 } } peers { permanent_uuid: "74fb0a58bbc04969a2fbe9c044f837ae" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 46159 } } peers { permanent_uuid: "157ae8bbaa2c41c8b96615e2f6f5cecc" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 46679 } }
I20260924 13:48:10.768502 22035 raft_consensus.cc:374] T 647f95b6f1f24fddbc52d68590da28c3 P 157ae8bbaa2c41c8b96615e2f6f5cecc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 13:48:10.768523 22035 raft_consensus.cc:729] T 647f95b6f1f24fddbc52d68590da28c3 P 157ae8bbaa2c41c8b96615e2f6f5cecc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 157ae8bbaa2c41c8b96615e2f6f5cecc, State: Initialized, Role: FOLLOWER
I20260924 13:48:10.768570 22035 consensus_queue.cc:260] T 647f95b6f1f24fddbc52d68590da28c3 P 157ae8bbaa2c41c8b96615e2f6f5cecc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a66912dfe2404677a7a20ec0462d8ff0" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 45785 } } peers { permanent_uuid: "74fb0a58bbc04969a2fbe9c044f837ae" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 46159 } } peers { permanent_uuid: "157ae8bbaa2c41c8b96615e2f6f5cecc" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 46679 } }
I20260924 13:48:10.768659 22035 ts_tablet_manager.cc:1428] T 647f95b6f1f24fddbc52d68590da28c3 P 157ae8bbaa2c41c8b96615e2f6f5cecc: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260924 13:48:10.769364 22034 tablet_bootstrap.cc:492] T 647f95b6f1f24fddbc52d68590da28c3 P 74fb0a58bbc04969a2fbe9c044f837ae: No bootstrap required, opened a new log
I20260924 13:48:10.769419 22034 ts_tablet_manager.cc:1397] T 647f95b6f1f24fddbc52d68590da28c3 P 74fb0a58bbc04969a2fbe9c044f837ae: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260924 13:48:10.769579 22034 raft_consensus.cc:348] T 647f95b6f1f24fddbc52d68590da28c3 P 74fb0a58bbc04969a2fbe9c044f837ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a66912dfe2404677a7a20ec0462d8ff0" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 45785 } } peers { permanent_uuid: "74fb0a58bbc04969a2fbe9c044f837ae" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 46159 } } peers { permanent_uuid: "157ae8bbaa2c41c8b96615e2f6f5cecc" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 46679 } }
I20260924 13:48:10.769644 22034 raft_consensus.cc:374] T 647f95b6f1f24fddbc52d68590da28c3 P 74fb0a58bbc04969a2fbe9c044f837ae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 13:48:10.769660 22034 raft_consensus.cc:729] T 647f95b6f1f24fddbc52d68590da28c3 P 74fb0a58bbc04969a2fbe9c044f837ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 74fb0a58bbc04969a2fbe9c044f837ae, State: Initialized, Role: FOLLOWER
I20260924 13:48:10.769697 22034 consensus_queue.cc:260] T 647f95b6f1f24fddbc52d68590da28c3 P 74fb0a58bbc04969a2fbe9c044f837ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a66912dfe2404677a7a20ec0462d8ff0" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 45785 } } peers { permanent_uuid: "74fb0a58bbc04969a2fbe9c044f837ae" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 46159 } } peers { permanent_uuid: "157ae8bbaa2c41c8b96615e2f6f5cecc" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 46679 } }
I20260924 13:48:10.769783 22034 ts_tablet_manager.cc:1428] T 647f95b6f1f24fddbc52d68590da28c3 P 74fb0a58bbc04969a2fbe9c044f837ae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260924 13:48:10.993057 22075 raft_consensus.cc:482] T 647f95b6f1f24fddbc52d68590da28c3 P a66912dfe2404677a7a20ec0462d8ff0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 13:48:10.993177 22075 raft_consensus.cc:504] T 647f95b6f1f24fddbc52d68590da28c3 P a66912dfe2404677a7a20ec0462d8ff0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a66912dfe2404677a7a20ec0462d8ff0" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 45785 } } peers { permanent_uuid: "74fb0a58bbc04969a2fbe9c044f837ae" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 46159 } } peers { permanent_uuid: "157ae8bbaa2c41c8b96615e2f6f5cecc" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 46679 } }
I20260924 13:48:10.993340 22075 leader_election.cc:288] T 647f95b6f1f24fddbc52d68590da28c3 P a66912dfe2404677a7a20ec0462d8ff0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 74fb0a58bbc04969a2fbe9c044f837ae (127.13.149.194:46159), 157ae8bbaa2c41c8b96615e2f6f5cecc (127.13.149.195:46679)
I20260924 13:48:10.993634 21960 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "647f95b6f1f24fddbc52d68590da28c3" candidate_uuid: "a66912dfe2404677a7a20ec0462d8ff0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "157ae8bbaa2c41c8b96615e2f6f5cecc" is_pre_election: true
I20260924 13:48:10.993736 21960 raft_consensus.cc:2454] T 647f95b6f1f24fddbc52d68590da28c3 P 157ae8bbaa2c41c8b96615e2f6f5cecc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a66912dfe2404677a7a20ec0462d8ff0 in term 0.
I20260924 13:48:10.993835 21820 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "647f95b6f1f24fddbc52d68590da28c3" candidate_uuid: "a66912dfe2404677a7a20ec0462d8ff0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74fb0a58bbc04969a2fbe9c044f837ae" is_pre_election: true
I20260924 13:48:10.993878 21611 leader_election.cc:302] T 647f95b6f1f24fddbc52d68590da28c3 P a66912dfe2404677a7a20ec0462d8ff0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 157ae8bbaa2c41c8b96615e2f6f5cecc, a66912dfe2404677a7a20ec0462d8ff0; no voters:
I20260924 13:48:10.993921 21820 raft_consensus.cc:2454] T 647f95b6f1f24fddbc52d68590da28c3 P 74fb0a58bbc04969a2fbe9c044f837ae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a66912dfe2404677a7a20ec0462d8ff0 in term 0.
I20260924 13:48:10.993965 22075 raft_consensus.cc:2781] T 647f95b6f1f24fddbc52d68590da28c3 P a66912dfe2404677a7a20ec0462d8ff0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 13:48:10.993996 22075 raft_consensus.cc:482] T 647f95b6f1f24fddbc52d68590da28c3 P a66912dfe2404677a7a20ec0462d8ff0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 13:48:10.994014 22075 raft_consensus.cc:3037] T 647f95b6f1f24fddbc52d68590da28c3 P a66912dfe2404677a7a20ec0462d8ff0 [term 0 FOLLOWER]: Advancing to term 1
I20260924 13:48:10.994721 22075 raft_consensus.cc:504] T 647f95b6f1f24fddbc52d68590da28c3 P a66912dfe2404677a7a20ec0462d8ff0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a66912dfe2404677a7a20ec0462d8ff0" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 45785 } } peers { permanent_uuid: "74fb0a58bbc04969a2fbe9c044f837ae" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 46159 } } peers { permanent_uuid: "157ae8bbaa2c41c8b96615e2f6f5cecc" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 46679 } }
I20260924 13:48:10.994825 22075 leader_election.cc:288] T 647f95b6f1f24fddbc52d68590da28c3 P a66912dfe2404677a7a20ec0462d8ff0 [CANDIDATE]: Term 1 election: Requested vote from peers 74fb0a58bbc04969a2fbe9c044f837ae (127.13.149.194:46159), 157ae8bbaa2c41c8b96615e2f6f5cecc (127.13.149.195:46679)
I20260924 13:48:10.995045 21820 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "647f95b6f1f24fddbc52d68590da28c3" candidate_uuid: "a66912dfe2404677a7a20ec0462d8ff0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74fb0a58bbc04969a2fbe9c044f837ae"
I20260924 13:48:10.995121 21820 raft_consensus.cc:3037] T 647f95b6f1f24fddbc52d68590da28c3 P 74fb0a58bbc04969a2fbe9c044f837ae [term 0 FOLLOWER]: Advancing to term 1
I20260924 13:48:10.995806 21820 raft_consensus.cc:2454] T 647f95b6f1f24fddbc52d68590da28c3 P 74fb0a58bbc04969a2fbe9c044f837ae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a66912dfe2404677a7a20ec0462d8ff0 in term 1.
I20260924 13:48:10.995927 21960 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "647f95b6f1f24fddbc52d68590da28c3" candidate_uuid: "a66912dfe2404677a7a20ec0462d8ff0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "157ae8bbaa2c41c8b96615e2f6f5cecc"
I20260924 13:48:10.995970 21611 leader_election.cc:302] T 647f95b6f1f24fddbc52d68590da28c3 P a66912dfe2404677a7a20ec0462d8ff0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74fb0a58bbc04969a2fbe9c044f837ae, a66912dfe2404677a7a20ec0462d8ff0; no voters:
I20260924 13:48:10.996021 21960 raft_consensus.cc:3037] T 647f95b6f1f24fddbc52d68590da28c3 P 157ae8bbaa2c41c8b96615e2f6f5cecc [term 0 FOLLOWER]: Advancing to term 1
I20260924 13:48:10.996115 22075 raft_consensus.cc:2781] T 647f95b6f1f24fddbc52d68590da28c3 P a66912dfe2404677a7a20ec0462d8ff0 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 13:48:10.996181 22075 raft_consensus.cc:686] T 647f95b6f1f24fddbc52d68590da28c3 P a66912dfe2404677a7a20ec0462d8ff0 [term 1 LEADER]: Becoming Leader. State: Replica: a66912dfe2404677a7a20ec0462d8ff0, State: Running, Role: LEADER
I20260924 13:48:10.996235 22075 consensus_queue.cc:237] T 647f95b6f1f24fddbc52d68590da28c3 P a66912dfe2404677a7a20ec0462d8ff0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a66912dfe2404677a7a20ec0462d8ff0" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 45785 } } peers { permanent_uuid: "74fb0a58bbc04969a2fbe9c044f837ae" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 46159 } } peers { permanent_uuid: "157ae8bbaa2c41c8b96615e2f6f5cecc" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 46679 } }
I20260924 13:48:10.996809 21960 raft_consensus.cc:2454] T 647f95b6f1f24fddbc52d68590da28c3 P 157ae8bbaa2c41c8b96615e2f6f5cecc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a66912dfe2404677a7a20ec0462d8ff0 in term 1.
I20260924 13:48:10.996819 21523 catalog_manager.cc:5495] T 647f95b6f1f24fddbc52d68590da28c3 P a66912dfe2404677a7a20ec0462d8ff0 reported cstate change: term changed from 0 to 1, leader changed from <none> to a66912dfe2404677a7a20ec0462d8ff0 (127.13.149.193). New cstate: current_term: 1 leader_uuid: "a66912dfe2404677a7a20ec0462d8ff0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a66912dfe2404677a7a20ec0462d8ff0" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 45785 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "74fb0a58bbc04969a2fbe9c044f837ae" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 46159 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "157ae8bbaa2c41c8b96615e2f6f5cecc" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 46679 } health_report { overall_health: UNKNOWN } } }
I20260924 13:48:11.056252 22030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:48:11.054110 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34918 (local address 127.13.149.193:45785)
0924 13:48:11.054169 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0924 13:48:11.054171 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:48:11.054193 (+ 22us) server_negotiation.cc:407] Connection header received
0924 13:48:11.054248 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:48:11.054248 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:48:11.054257 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:48:11.054296 (+ 39us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 13:48:11.054422 (+ 126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:48:11.054892 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:48:11.055623 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:48:11.055842 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:48:11.055949 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:48:11.056049 (+ 100us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 13:48:11.056077 (+ 28us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 13:48:11.056111 (+ 34us) server_negotiation.cc:1035] Waiting for connection context
0924 13:48:11.056155 (+ 44us) server_negotiation.cc:299] Negotiation successful
0924 13:48:11.056173 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260924 13:48:11.057089 21960 raft_consensus.cc:1261] T 647f95b6f1f24fddbc52d68590da28c3 P 157ae8bbaa2c41c8b96615e2f6f5cecc [term 1 FOLLOWER]: Refusing update from remote peer a66912dfe2404677a7a20ec0462d8ff0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 13:48:11.057101 21820 raft_consensus.cc:1261] T 647f95b6f1f24fddbc52d68590da28c3 P 74fb0a58bbc04969a2fbe9c044f837ae [term 1 FOLLOWER]: Refusing update from remote peer a66912dfe2404677a7a20ec0462d8ff0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 13:48:11.057327 22075 consensus_queue.cc:1035] T 647f95b6f1f24fddbc52d68590da28c3 P a66912dfe2404677a7a20ec0462d8ff0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "74fb0a58bbc04969a2fbe9c044f837ae" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 46159 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 13:48:11.057440 22086 consensus_queue.cc:1035] T 647f95b6f1f24fddbc52d68590da28c3 P a66912dfe2404677a7a20ec0462d8ff0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "157ae8bbaa2c41c8b96615e2f6f5cecc" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 46679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 13:48:11.059995 21523 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46490:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 13:48:11.060088 21523 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46490:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 13:48:11.060662 21523 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9f6c1d2f544b490eb7522be8ff50ceba: Sending DeleteTablet for 3 replicas of tablet 647f95b6f1f24fddbc52d68590da28c3
I20260924 13:48:11.060878 21660 tablet_service.cc:1514] Processing DeleteTablet for tablet 647f95b6f1f24fddbc52d68590da28c3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 13:48:11 UTC) from {username='oryx', principal='oryx/127.13.149.254@KRBTEST.COM'} at 127.0.0.1:34888
I20260924 13:48:11.060967 21800 tablet_service.cc:1514] Processing DeleteTablet for tablet 647f95b6f1f24fddbc52d68590da28c3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 13:48:11 UTC) from {username='oryx', principal='oryx/127.13.149.254@KRBTEST.COM'} at 127.0.0.1:47830
I20260924 13:48:11.060987 21940 tablet_service.cc:1514] Processing DeleteTablet for tablet 647f95b6f1f24fddbc52d68590da28c3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 13:48:11 UTC) from {username='oryx', principal='oryx/127.13.149.254@KRBTEST.COM'} at 127.0.0.1:42746
I20260924 13:48:11.061021 22104 tablet_replica.cc:330] T 647f95b6f1f24fddbc52d68590da28c3 P a66912dfe2404677a7a20ec0462d8ff0: stopping tablet replica
I20260924 13:48:11.061055 22103 tablet_replica.cc:330] T 647f95b6f1f24fddbc52d68590da28c3 P 74fb0a58bbc04969a2fbe9c044f837ae: stopping tablet replica
I20260924 13:48:11.061074 22102 tablet_replica.cc:330] T 647f95b6f1f24fddbc52d68590da28c3 P 157ae8bbaa2c41c8b96615e2f6f5cecc: stopping tablet replica
I20260924 13:48:11.061090 22103 raft_consensus.cc:2229] T 647f95b6f1f24fddbc52d68590da28c3 P 74fb0a58bbc04969a2fbe9c044f837ae [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 13:48:11.061089 22104 raft_consensus.cc:2229] T 647f95b6f1f24fddbc52d68590da28c3 P a66912dfe2404677a7a20ec0462d8ff0 [term 1 LEADER]: Raft consensus shutting down.
I20260924 13:48:11.061122 22102 raft_consensus.cc:2229] T 647f95b6f1f24fddbc52d68590da28c3 P 157ae8bbaa2c41c8b96615e2f6f5cecc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 13:48:11.061134 22103 raft_consensus.cc:2258] T 647f95b6f1f24fddbc52d68590da28c3 P 74fb0a58bbc04969a2fbe9c044f837ae [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 13:48:11.061146 22104 raft_consensus.cc:2258] T 647f95b6f1f24fddbc52d68590da28c3 P a66912dfe2404677a7a20ec0462d8ff0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 13:48:11.061168 22102 raft_consensus.cc:2258] T 647f95b6f1f24fddbc52d68590da28c3 P 157ae8bbaa2c41c8b96615e2f6f5cecc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 13:48:11.061244 21523 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46490:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 13:48:11.061383 21523 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 13:48:11.061455 22103 ts_tablet_manager.cc:1905] T 647f95b6f1f24fddbc52d68590da28c3 P 74fb0a58bbc04969a2fbe9c044f837ae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 13:48:11.062026 22102 ts_tablet_manager.cc:1905] T 647f95b6f1f24fddbc52d68590da28c3 P 157ae8bbaa2c41c8b96615e2f6f5cecc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 13:48:11.062742 21660 tablet_service.cc:1467] Processing CreateTablet for tablet 21643fd353ff47aebb3d4be300a652ef (DEFAULT_TABLE table=test-table [id=89523938f6da429bb944c7c1a148949b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 13:48:11.062826 21940 tablet_service.cc:1467] Processing CreateTablet for tablet 21643fd353ff47aebb3d4be300a652ef (DEFAULT_TABLE table=test-table [id=89523938f6da429bb944c7c1a148949b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 13:48:11.062856 21660 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 21643fd353ff47aebb3d4be300a652ef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 13:48:11.062932 21940 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 21643fd353ff47aebb3d4be300a652ef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 13:48:11.063068 22104 ts_tablet_manager.cc:1905] T 647f95b6f1f24fddbc52d68590da28c3 P a66912dfe2404677a7a20ec0462d8ff0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 13:48:11.063764 22102 ts_tablet_manager.cc:1918] T 647f95b6f1f24fddbc52d68590da28c3 P 157ae8bbaa2c41c8b96615e2f6f5cecc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 13:48:11.063819 22102 log.cc:1196] T 647f95b6f1f24fddbc52d68590da28c3 P 157ae8bbaa2c41c8b96615e2f6f5cecc: Deleting WAL directory at /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790257629041645-13911-0/minicluster-data/ts-2/wal/wals/647f95b6f1f24fddbc52d68590da28c3
I20260924 13:48:11.064029 22102 ts_tablet_manager.cc:1939] T 647f95b6f1f24fddbc52d68590da28c3 P 157ae8bbaa2c41c8b96615e2f6f5cecc: Deleting consensus metadata
I20260924 13:48:11.064190 22037 tablet_bootstrap.cc:492] T 21643fd353ff47aebb3d4be300a652ef P a66912dfe2404677a7a20ec0462d8ff0: Bootstrap starting.
I20260924 13:48:11.064412 22035 tablet_bootstrap.cc:492] T 21643fd353ff47aebb3d4be300a652ef P 157ae8bbaa2c41c8b96615e2f6f5cecc: Bootstrap starting.
I20260924 13:48:11.064440 22037 tablet_bootstrap.cc:654] T 21643fd353ff47aebb3d4be300a652ef P a66912dfe2404677a7a20ec0462d8ff0: Neither blocks nor log segments found. Creating new log.
I20260924 13:48:11.064441 22104 ts_tablet_manager.cc:1918] T 647f95b6f1f24fddbc52d68590da28c3 P a66912dfe2404677a7a20ec0462d8ff0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 13:48:11.064560 22104 log.cc:1196] T 647f95b6f1f24fddbc52d68590da28c3 P a66912dfe2404677a7a20ec0462d8ff0: Deleting WAL directory at /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790257629041645-13911-0/minicluster-data/ts-0/wal/wals/647f95b6f1f24fddbc52d68590da28c3
I20260924 13:48:11.064514 22103 ts_tablet_manager.cc:1918] T 647f95b6f1f24fddbc52d68590da28c3 P 74fb0a58bbc04969a2fbe9c044f837ae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 13:48:11.064595 22103 log.cc:1196] T 647f95b6f1f24fddbc52d68590da28c3 P 74fb0a58bbc04969a2fbe9c044f837ae: Deleting WAL directory at /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790257629041645-13911-0/minicluster-data/ts-1/wal/wals/647f95b6f1f24fddbc52d68590da28c3
I20260924 13:48:11.064715 22035 tablet_bootstrap.cc:654] T 21643fd353ff47aebb3d4be300a652ef P 157ae8bbaa2c41c8b96615e2f6f5cecc: Neither blocks nor log segments found. Creating new log.
I20260924 13:48:11.064755 22104 ts_tablet_manager.cc:1939] T 647f95b6f1f24fddbc52d68590da28c3 P a66912dfe2404677a7a20ec0462d8ff0: Deleting consensus metadata
I20260924 13:48:11.064817 22103 ts_tablet_manager.cc:1939] T 647f95b6f1f24fddbc52d68590da28c3 P 74fb0a58bbc04969a2fbe9c044f837ae: Deleting consensus metadata
I20260924 13:48:11.064981 21800 tablet_service.cc:1467] Processing CreateTablet for tablet 21643fd353ff47aebb3d4be300a652ef (DEFAULT_TABLE table=test-table [id=89523938f6da429bb944c7c1a148949b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 13:48:11.065080 21800 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 21643fd353ff47aebb3d4be300a652ef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 13:48:11.065081 21507 catalog_manager.cc:4841] TS 157ae8bbaa2c41c8b96615e2f6f5cecc (127.13.149.195:46679): tablet 647f95b6f1f24fddbc52d68590da28c3 (table test-table [id=76844b46e06642e192a32dc5e14ec6db]) successfully deleted
I20260924 13:48:11.065172 21507 catalog_manager.cc:4841] TS a66912dfe2404677a7a20ec0462d8ff0 (127.13.149.193:45785): tablet 647f95b6f1f24fddbc52d68590da28c3 (table test-table [id=76844b46e06642e192a32dc5e14ec6db]) successfully deleted
I20260924 13:48:11.065430 22037 tablet_bootstrap.cc:492] T 21643fd353ff47aebb3d4be300a652ef P a66912dfe2404677a7a20ec0462d8ff0: No bootstrap required, opened a new log
I20260924 13:48:11.065470 22037 ts_tablet_manager.cc:1397] T 21643fd353ff47aebb3d4be300a652ef P a66912dfe2404677a7a20ec0462d8ff0: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260924 13:48:11.065572 22037 raft_consensus.cc:348] T 21643fd353ff47aebb3d4be300a652ef P a66912dfe2404677a7a20ec0462d8ff0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a66912dfe2404677a7a20ec0462d8ff0" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 45785 } } peers { permanent_uuid: "74fb0a58bbc04969a2fbe9c044f837ae" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 46159 } } peers { permanent_uuid: "157ae8bbaa2c41c8b96615e2f6f5cecc" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 46679 } }
I20260924 13:48:11.065635 22037 raft_consensus.cc:374] T 21643fd353ff47aebb3d4be300a652ef P a66912dfe2404677a7a20ec0462d8ff0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 13:48:11.065654 22037 raft_consensus.cc:729] T 21643fd353ff47aebb3d4be300a652ef P a66912dfe2404677a7a20ec0462d8ff0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a66912dfe2404677a7a20ec0462d8ff0, State: Initialized, Role: FOLLOWER
I20260924 13:48:11.065703 22037 consensus_queue.cc:260] T 21643fd353ff47aebb3d4be300a652ef P a66912dfe2404677a7a20ec0462d8ff0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a66912dfe2404677a7a20ec0462d8ff0" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 45785 } } peers { permanent_uuid: "74fb0a58bbc04969a2fbe9c044f837ae" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 46159 } } peers { permanent_uuid: "157ae8bbaa2c41c8b96615e2f6f5cecc" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 46679 } }
I20260924 13:48:11.065793 22037 ts_tablet_manager.cc:1428] T 21643fd353ff47aebb3d4be300a652ef P a66912dfe2404677a7a20ec0462d8ff0: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260924 13:48:11.066239 22034 tablet_bootstrap.cc:492] T 21643fd353ff47aebb3d4be300a652ef P 74fb0a58bbc04969a2fbe9c044f837ae: Bootstrap starting.
I20260924 13:48:11.066290 21507 catalog_manager.cc:4841] TS 74fb0a58bbc04969a2fbe9c044f837ae (127.13.149.194:46159): tablet 647f95b6f1f24fddbc52d68590da28c3 (table test-table [id=76844b46e06642e192a32dc5e14ec6db]) successfully deleted
I20260924 13:48:11.066601 22034 tablet_bootstrap.cc:654] T 21643fd353ff47aebb3d4be300a652ef P 74fb0a58bbc04969a2fbe9c044f837ae: Neither blocks nor log segments found. Creating new log.
I20260924 13:48:11.067112 22035 tablet_bootstrap.cc:492] T 21643fd353ff47aebb3d4be300a652ef P 157ae8bbaa2c41c8b96615e2f6f5cecc: No bootstrap required, opened a new log
I20260924 13:48:11.067157 22035 ts_tablet_manager.cc:1397] T 21643fd353ff47aebb3d4be300a652ef P 157ae8bbaa2c41c8b96615e2f6f5cecc: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20260924 13:48:11.067158 22034 tablet_bootstrap.cc:492] T 21643fd353ff47aebb3d4be300a652ef P 74fb0a58bbc04969a2fbe9c044f837ae: No bootstrap required, opened a new log
I20260924 13:48:11.067198 22034 ts_tablet_manager.cc:1397] T 21643fd353ff47aebb3d4be300a652ef P 74fb0a58bbc04969a2fbe9c044f837ae: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260924 13:48:11.067239 22035 raft_consensus.cc:348] T 21643fd353ff47aebb3d4be300a652ef P 157ae8bbaa2c41c8b96615e2f6f5cecc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a66912dfe2404677a7a20ec0462d8ff0" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 45785 } } peers { permanent_uuid: "74fb0a58bbc04969a2fbe9c044f837ae" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 46159 } } peers { permanent_uuid: "157ae8bbaa2c41c8b96615e2f6f5cecc" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 46679 } }
I20260924 13:48:11.067294 22035 raft_consensus.cc:374] T 21643fd353ff47aebb3d4be300a652ef P 157ae8bbaa2c41c8b96615e2f6f5cecc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 13:48:11.067306 22035 raft_consensus.cc:729] T 21643fd353ff47aebb3d4be300a652ef P 157ae8bbaa2c41c8b96615e2f6f5cecc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 157ae8bbaa2c41c8b96615e2f6f5cecc, State: Initialized, Role: FOLLOWER
I20260924 13:48:11.067342 22035 consensus_queue.cc:260] T 21643fd353ff47aebb3d4be300a652ef P 157ae8bbaa2c41c8b96615e2f6f5cecc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a66912dfe2404677a7a20ec0462d8ff0" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 45785 } } peers { permanent_uuid: "74fb0a58bbc04969a2fbe9c044f837ae" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 46159 } } peers { permanent_uuid: "157ae8bbaa2c41c8b96615e2f6f5cecc" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 46679 } }
I20260924 13:48:11.067333 22034 raft_consensus.cc:348] T 21643fd353ff47aebb3d4be300a652ef P 74fb0a58bbc04969a2fbe9c044f837ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a66912dfe2404677a7a20ec0462d8ff0" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 45785 } } peers { permanent_uuid: "74fb0a58bbc04969a2fbe9c044f837ae" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 46159 } } peers { permanent_uuid: "157ae8bbaa2c41c8b96615e2f6f5cecc" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 46679 } }
I20260924 13:48:11.067404 22034 raft_consensus.cc:374] T 21643fd353ff47aebb3d4be300a652ef P 74fb0a58bbc04969a2fbe9c044f837ae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 13:48:11.067426 22034 raft_consensus.cc:729] T 21643fd353ff47aebb3d4be300a652ef P 74fb0a58bbc04969a2fbe9c044f837ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 74fb0a58bbc04969a2fbe9c044f837ae, State: Initialized, Role: FOLLOWER
I20260924 13:48:11.067432 22035 ts_tablet_manager.cc:1428] T 21643fd353ff47aebb3d4be300a652ef P 157ae8bbaa2c41c8b96615e2f6f5cecc: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260924 13:48:11.067462 22034 consensus_queue.cc:260] T 21643fd353ff47aebb3d4be300a652ef P 74fb0a58bbc04969a2fbe9c044f837ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a66912dfe2404677a7a20ec0462d8ff0" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 45785 } } peers { permanent_uuid: "74fb0a58bbc04969a2fbe9c044f837ae" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 46159 } } peers { permanent_uuid: "157ae8bbaa2c41c8b96615e2f6f5cecc" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 46679 } }
I20260924 13:48:11.067546 22034 ts_tablet_manager.cc:1428] T 21643fd353ff47aebb3d4be300a652ef P 74fb0a58bbc04969a2fbe9c044f837ae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260924 13:48:11.237630 22053 raft_consensus.cc:482] T 21643fd353ff47aebb3d4be300a652ef P a66912dfe2404677a7a20ec0462d8ff0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 13:48:11.237744 22053 raft_consensus.cc:504] T 21643fd353ff47aebb3d4be300a652ef P a66912dfe2404677a7a20ec0462d8ff0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a66912dfe2404677a7a20ec0462d8ff0" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 45785 } } peers { permanent_uuid: "74fb0a58bbc04969a2fbe9c044f837ae" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 46159 } } peers { permanent_uuid: "157ae8bbaa2c41c8b96615e2f6f5cecc" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 46679 } }
I20260924 13:48:11.237879 22053 leader_election.cc:288] T 21643fd353ff47aebb3d4be300a652ef P a66912dfe2404677a7a20ec0462d8ff0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 74fb0a58bbc04969a2fbe9c044f837ae (127.13.149.194:46159), 157ae8bbaa2c41c8b96615e2f6f5cecc (127.13.149.195:46679)
I20260924 13:48:11.238217 21820 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "21643fd353ff47aebb3d4be300a652ef" candidate_uuid: "a66912dfe2404677a7a20ec0462d8ff0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74fb0a58bbc04969a2fbe9c044f837ae" is_pre_election: true
I20260924 13:48:11.238231 21960 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "21643fd353ff47aebb3d4be300a652ef" candidate_uuid: "a66912dfe2404677a7a20ec0462d8ff0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "157ae8bbaa2c41c8b96615e2f6f5cecc" is_pre_election: true
I20260924 13:48:11.238317 21820 raft_consensus.cc:2454] T 21643fd353ff47aebb3d4be300a652ef P 74fb0a58bbc04969a2fbe9c044f837ae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a66912dfe2404677a7a20ec0462d8ff0 in term 0.
I20260924 13:48:11.238327 21960 raft_consensus.cc:2454] T 21643fd353ff47aebb3d4be300a652ef P 157ae8bbaa2c41c8b96615e2f6f5cecc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a66912dfe2404677a7a20ec0462d8ff0 in term 0.
I20260924 13:48:11.238513 21611 leader_election.cc:302] T 21643fd353ff47aebb3d4be300a652ef P a66912dfe2404677a7a20ec0462d8ff0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74fb0a58bbc04969a2fbe9c044f837ae, a66912dfe2404677a7a20ec0462d8ff0; no voters:
I20260924 13:48:11.238613 22053 raft_consensus.cc:2781] T 21643fd353ff47aebb3d4be300a652ef P a66912dfe2404677a7a20ec0462d8ff0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 13:48:11.238655 22053 raft_consensus.cc:482] T 21643fd353ff47aebb3d4be300a652ef P a66912dfe2404677a7a20ec0462d8ff0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 13:48:11.238674 22053 raft_consensus.cc:3037] T 21643fd353ff47aebb3d4be300a652ef P a66912dfe2404677a7a20ec0462d8ff0 [term 0 FOLLOWER]: Advancing to term 1
I20260924 13:48:11.239323 22053 raft_consensus.cc:504] T 21643fd353ff47aebb3d4be300a652ef P a66912dfe2404677a7a20ec0462d8ff0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a66912dfe2404677a7a20ec0462d8ff0" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 45785 } } peers { permanent_uuid: "74fb0a58bbc04969a2fbe9c044f837ae" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 46159 } } peers { permanent_uuid: "157ae8bbaa2c41c8b96615e2f6f5cecc" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 46679 } }
I20260924 13:48:11.239439 22053 leader_election.cc:288] T 21643fd353ff47aebb3d4be300a652ef P a66912dfe2404677a7a20ec0462d8ff0 [CANDIDATE]: Term 1 election: Requested vote from peers 74fb0a58bbc04969a2fbe9c044f837ae (127.13.149.194:46159), 157ae8bbaa2c41c8b96615e2f6f5cecc (127.13.149.195:46679)
I20260924 13:48:11.239631 21960 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "21643fd353ff47aebb3d4be300a652ef" candidate_uuid: "a66912dfe2404677a7a20ec0462d8ff0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "157ae8bbaa2c41c8b96615e2f6f5cecc"
I20260924 13:48:11.239632 21820 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "21643fd353ff47aebb3d4be300a652ef" candidate_uuid: "a66912dfe2404677a7a20ec0462d8ff0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74fb0a58bbc04969a2fbe9c044f837ae"
I20260924 13:48:11.239708 21820 raft_consensus.cc:3037] T 21643fd353ff47aebb3d4be300a652ef P 74fb0a58bbc04969a2fbe9c044f837ae [term 0 FOLLOWER]: Advancing to term 1
I20260924 13:48:11.239707 21960 raft_consensus.cc:3037] T 21643fd353ff47aebb3d4be300a652ef P 157ae8bbaa2c41c8b96615e2f6f5cecc [term 0 FOLLOWER]: Advancing to term 1
I20260924 13:48:11.240407 21820 raft_consensus.cc:2454] T 21643fd353ff47aebb3d4be300a652ef P 74fb0a58bbc04969a2fbe9c044f837ae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a66912dfe2404677a7a20ec0462d8ff0 in term 1.
I20260924 13:48:11.240410 21960 raft_consensus.cc:2454] T 21643fd353ff47aebb3d4be300a652ef P 157ae8bbaa2c41c8b96615e2f6f5cecc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a66912dfe2404677a7a20ec0462d8ff0 in term 1.
I20260924 13:48:11.240609 21611 leader_election.cc:302] T 21643fd353ff47aebb3d4be300a652ef P a66912dfe2404677a7a20ec0462d8ff0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74fb0a58bbc04969a2fbe9c044f837ae, a66912dfe2404677a7a20ec0462d8ff0; no voters:
I20260924 13:48:11.240705 22053 raft_consensus.cc:2781] T 21643fd353ff47aebb3d4be300a652ef P a66912dfe2404677a7a20ec0462d8ff0 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 13:48:11.240757 22053 raft_consensus.cc:686] T 21643fd353ff47aebb3d4be300a652ef P a66912dfe2404677a7a20ec0462d8ff0 [term 1 LEADER]: Becoming Leader. State: Replica: a66912dfe2404677a7a20ec0462d8ff0, State: Running, Role: LEADER
I20260924 13:48:11.240805 22053 consensus_queue.cc:237] T 21643fd353ff47aebb3d4be300a652ef P a66912dfe2404677a7a20ec0462d8ff0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a66912dfe2404677a7a20ec0462d8ff0" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 45785 } } peers { permanent_uuid: "74fb0a58bbc04969a2fbe9c044f837ae" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 46159 } } peers { permanent_uuid: "157ae8bbaa2c41c8b96615e2f6f5cecc" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 46679 } }
I20260924 13:48:11.241572 21523 catalog_manager.cc:5495] T 21643fd353ff47aebb3d4be300a652ef P a66912dfe2404677a7a20ec0462d8ff0 reported cstate change: term changed from 0 to 1, leader changed from <none> to a66912dfe2404677a7a20ec0462d8ff0 (127.13.149.193). New cstate: current_term: 1 leader_uuid: "a66912dfe2404677a7a20ec0462d8ff0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a66912dfe2404677a7a20ec0462d8ff0" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 45785 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "74fb0a58bbc04969a2fbe9c044f837ae" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 46159 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "157ae8bbaa2c41c8b96615e2f6f5cecc" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 46679 } health_report { overall_health: UNKNOWN } } }
I20260924 13:48:11.255545 22030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:48:11.253125 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.149.193:44745 (local address 127.13.149.193:45785)
0924 13:48:11.253173 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0924 13:48:11.253175 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:48:11.253223 (+ 48us) server_negotiation.cc:407] Connection header received
0924 13:48:11.253271 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:48:11.253272 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:48:11.253280 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:48:11.253338 (+ 58us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 13:48:11.253431 (+ 93us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:48:11.253924 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:48:11.254720 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:48:11.254912 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:48:11.254975 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:48:11.255156 (+ 181us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 13:48:11.255188 (+ 32us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 13:48:11.255303 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0924 13:48:11.255366 (+ 63us) server_negotiation.cc:299] Negotiation successful
0924 13:48:11.255390 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":21}
I20260924 13:48:11.255559 22130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:48:11.253075 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.193:45785 (local address 127.13.149.193:44745)
0924 13:48:11.253165 (+ 90us) negotiation.cc:107] Waiting for socket to connect
0924 13:48:11.253175 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 13:48:11.253234 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:48:11.253341 (+ 107us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:48:11.253341 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:48:11.253343 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0924 13:48:11.253397 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:48:11.253398 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:48:11.254029 (+ 631us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:48:11.254031 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:48:11.254602 (+ 571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:48:11.254603 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:48:11.255006 (+ 403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:48:11.255007 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:48:11.255053 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:48:11.255058 (+ 5us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0924 13:48:11.255302 (+ 244us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0924 13:48:11.255305 (+ 3us) client_negotiation.cc:769] Sending connection context
0924 13:48:11.255346 (+ 41us) client_negotiation.cc:240] Negotiation successful
0924 13:48:11.255476 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":70,"mutex_wait_us":3,"thread_start_us":38,"threads_started":1}
I20260924 13:48:11.257737 21960 raft_consensus.cc:1261] T 21643fd353ff47aebb3d4be300a652ef P 157ae8bbaa2c41c8b96615e2f6f5cecc [term 1 FOLLOWER]: Refusing update from remote peer a66912dfe2404677a7a20ec0462d8ff0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 13:48:11.257762 21820 raft_consensus.cc:1261] T 21643fd353ff47aebb3d4be300a652ef P 74fb0a58bbc04969a2fbe9c044f837ae [term 1 FOLLOWER]: Refusing update from remote peer a66912dfe2404677a7a20ec0462d8ff0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 13:48:11.258042 22086 consensus_queue.cc:1035] T 21643fd353ff47aebb3d4be300a652ef P a66912dfe2404677a7a20ec0462d8ff0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "74fb0a58bbc04969a2fbe9c044f837ae" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 46159 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 13:48:11.258163 22053 consensus_queue.cc:1035] T 21643fd353ff47aebb3d4be300a652ef P a66912dfe2404677a7a20ec0462d8ff0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "157ae8bbaa2c41c8b96615e2f6f5cecc" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 46679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 13:48:11.270555 21522 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46490:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 13:48:11.270668 21522 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46490:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 13:48:11.271368 21522 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9f6c1d2f544b490eb7522be8ff50ceba: Sending DeleteTablet for 3 replicas of tablet 21643fd353ff47aebb3d4be300a652ef
I20260924 13:48:11.271648 21659 tablet_service.cc:1514] Processing DeleteTablet for tablet 21643fd353ff47aebb3d4be300a652ef with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 13:48:11 UTC) from {username='oryx', principal='oryx/127.13.149.254@KRBTEST.COM'} at 127.0.0.1:34888
I20260924 13:48:11.271708 21940 tablet_service.cc:1514] Processing DeleteTablet for tablet 21643fd353ff47aebb3d4be300a652ef with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 13:48:11 UTC) from {username='oryx', principal='oryx/127.13.149.254@KRBTEST.COM'} at 127.0.0.1:42746
I20260924 13:48:11.271713 21800 tablet_service.cc:1514] Processing DeleteTablet for tablet 21643fd353ff47aebb3d4be300a652ef with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 13:48:11 UTC) from {username='oryx', principal='oryx/127.13.149.254@KRBTEST.COM'} at 127.0.0.1:47830
I20260924 13:48:11.271772 22104 tablet_replica.cc:330] T 21643fd353ff47aebb3d4be300a652ef P a66912dfe2404677a7a20ec0462d8ff0: stopping tablet replica
I20260924 13:48:11.271788 22102 tablet_replica.cc:330] T 21643fd353ff47aebb3d4be300a652ef P 157ae8bbaa2c41c8b96615e2f6f5cecc: stopping tablet replica
I20260924 13:48:11.271808 22103 tablet_replica.cc:330] T 21643fd353ff47aebb3d4be300a652ef P 74fb0a58bbc04969a2fbe9c044f837ae: stopping tablet replica
I20260924 13:48:11.271833 22102 raft_consensus.cc:2229] T 21643fd353ff47aebb3d4be300a652ef P 157ae8bbaa2c41c8b96615e2f6f5cecc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 13:48:11.271833 22104 raft_consensus.cc:2229] T 21643fd353ff47aebb3d4be300a652ef P a66912dfe2404677a7a20ec0462d8ff0 [term 1 LEADER]: Raft consensus shutting down.
I20260924 13:48:11.271852 22103 raft_consensus.cc:2229] T 21643fd353ff47aebb3d4be300a652ef P 74fb0a58bbc04969a2fbe9c044f837ae [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 13:48:11.271874 22102 raft_consensus.cc:2258] T 21643fd353ff47aebb3d4be300a652ef P 157ae8bbaa2c41c8b96615e2f6f5cecc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 13:48:11.271884 22103 raft_consensus.cc:2258] T 21643fd353ff47aebb3d4be300a652ef P 74fb0a58bbc04969a2fbe9c044f837ae [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 13:48:11.271895 22104 raft_consensus.cc:2258] T 21643fd353ff47aebb3d4be300a652ef P a66912dfe2404677a7a20ec0462d8ff0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 13:48:11.272248 22102 ts_tablet_manager.cc:1905] T 21643fd353ff47aebb3d4be300a652ef P 157ae8bbaa2c41c8b96615e2f6f5cecc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 13:48:11.272352 22104 ts_tablet_manager.cc:1905] T 21643fd353ff47aebb3d4be300a652ef P a66912dfe2404677a7a20ec0462d8ff0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 13:48:11.273554 13911 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJN9WRF/build/release/bin/kudu with pid 21595
I20260924 13:48:11.273768 22103 ts_tablet_manager.cc:1905] T 21643fd353ff47aebb3d4be300a652ef P 74fb0a58bbc04969a2fbe9c044f837ae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 13:48:11.274257 22102 ts_tablet_manager.cc:1918] T 21643fd353ff47aebb3d4be300a652ef P 157ae8bbaa2c41c8b96615e2f6f5cecc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260924 13:48:11.274309 22102 log.cc:1196] T 21643fd353ff47aebb3d4be300a652ef P 157ae8bbaa2c41c8b96615e2f6f5cecc: Deleting WAL directory at /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790257629041645-13911-0/minicluster-data/ts-2/wal/wals/21643fd353ff47aebb3d4be300a652ef
I20260924 13:48:11.274510 22102 ts_tablet_manager.cc:1939] T 21643fd353ff47aebb3d4be300a652ef P 157ae8bbaa2c41c8b96615e2f6f5cecc: Deleting consensus metadata
I20260924 13:48:11.274842 21507 catalog_manager.cc:4841] TS 157ae8bbaa2c41c8b96615e2f6f5cecc (127.13.149.195:46679): tablet 21643fd353ff47aebb3d4be300a652ef (table test-table [id=89523938f6da429bb944c7c1a148949b]) successfully deleted
I20260924 13:48:11.275149 22103 ts_tablet_manager.cc:1918] T 21643fd353ff47aebb3d4be300a652ef P 74fb0a58bbc04969a2fbe9c044f837ae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260924 13:48:11.275198 22103 log.cc:1196] T 21643fd353ff47aebb3d4be300a652ef P 74fb0a58bbc04969a2fbe9c044f837ae: Deleting WAL directory at /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790257629041645-13911-0/minicluster-data/ts-1/wal/wals/21643fd353ff47aebb3d4be300a652ef
I20260924 13:48:11.275378 22103 ts_tablet_manager.cc:1939] T 21643fd353ff47aebb3d4be300a652ef P 74fb0a58bbc04969a2fbe9c044f837ae: Deleting consensus metadata
I20260924 13:48:11.275755 21507 catalog_manager.cc:4841] TS 74fb0a58bbc04969a2fbe9c044f837ae (127.13.149.194:46159): tablet 21643fd353ff47aebb3d4be300a652ef (table test-table [id=89523938f6da429bb944c7c1a148949b]) successfully deleted
W20260924 13:48:11.280593 21507 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.13.149.193:45785 (error 108)
I20260924 13:48:11.280755 13911 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJN9WRF/build/release/bin/kudu with pid 21735
I20260924 13:48:11.281641 22135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:48:11.281380 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.193:45785 (local address 127.0.0.1:34926)
0924 13:48:11.281520 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0924 13:48:11.281590 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.149.193:45785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":47,"threads_started":1}
W20260924 13:48:11.281806 21507 catalog_manager.cc:4584] TS a66912dfe2404677a7a20ec0462d8ff0 (127.13.149.193:45785): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 21643fd353ff47aebb3d4be300a652ef: Network error: Client connection negotiation failed: client connection to 127.13.149.193:45785: connect: Connection refused (error 111)
I20260924 13:48:11.286121 13911 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJN9WRF/build/release/bin/kudu with pid 21875
I20260924 13:48:11.291574 13911 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJN9WRF/build/release/bin/kudu with pid 21491
2026-09-24T13:48:11Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (3141 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790257629041645-13911-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 13:48:11 dist-test-slave-sjl2 krb5kdc[22138](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 13:48:11 dist-test-slave-sjl2 krb5kdc[22138](info): set up 2 sockets
Sep 24 13:48:11 dist-test-slave-sjl2 krb5kdc[22138](info): commencing operation
krb5kdc: starting...
W20260924 13:48:12.254861 13911 mini_kdc.cc:121] Time spent starting KDC: real 0.943s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790257629041645-13911-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790257629041645-13911-0/krb5kdc/test-user.keytab.
Sep 24 13:48:12 dist-test-slave-sjl2 krb5kdc[22138](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257692, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T13:48:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T13:48:12Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.149.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.149.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.149.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.254.keytab.
Entry for principal kudu/127.13.149.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.254.keytab.
I20260924 13:48:12.380805 13911 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJN9WRF/build/release/bin/kudu
/tmp/dist-test-taskJN9WRF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790257629041645-13911-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790257629041645-13911-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790257629041645-13911-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790257629041645-13911-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.149.254:38135
--webserver_interface=127.13.149.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.254.keytab
--principal=kudu/127.13.149.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.149.212:40663
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.149.254:38135
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790257629041645-13911-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790257629041645-13911-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790257629041645-13911-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 13:48:12.463204 22151 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 13:48:12.463466 22151 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 13:48:12.463505 22151 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 13:48:12.465045 22151 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 13:48:12.465153 22151 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 13:48:12.465178 22151 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 13:48:12.465198 22151 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 13:48:12.465214 22151 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 13:48:12.468109 22151 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.149.212:40663
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790257629041645-13911-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790257629041645-13911-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.149.254:38135
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.254.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.149.254:38135
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790257629041645-13911-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.149.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22151
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790257629041645-13911-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 12:29:37 UTC on ccycloud.quasar-iqzbxn.root.comops.site
build id 4670106
I20260924 13:48:12.468484 22151 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 13:48:12.468739 22151 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 13:48:12.473616 22159 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 13:48:12.473688 22157 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 13:48:12.473826 22151 server_base.cc:1034] running on GCE node
W20260924 13:48:12.473616 22156 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 13:48:12.474221 22151 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 13:48:12.474539 22151 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 13:48:12.475720 22151 hybrid_clock.cc:648] HybridClock initialized: now 1790257692475699 us; error 36 us; skew 500 ppm
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-09-24T13:48:12Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1187 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790257629041645-13911-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 13:48:12 dist-test-slave-sjl2 krb5kdc[22164](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 13:48:12 dist-test-slave-sjl2 krb5kdc[22164](info): set up 2 sockets
Sep 24 13:48:12 dist-test-slave-sjl2 krb5kdc[22164](info): commencing operation
krb5kdc: starting...
W20260924 13:48:13.434876 13911 mini_kdc.cc:121] Time spent starting KDC: real 0.936s user 0.003s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790257629041645-13911-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790257629041645-13911-0/krb5kdc/test-user.keytab.
Sep 24 13:48:13 dist-test-slave-sjl2 krb5kdc[22164](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257693, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T13:48:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T13:48:13Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.149.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.149.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.149.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.254.keytab.
Entry for principal kudu/127.13.149.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.254.keytab.
I20260924 13:48:13.560652 13911 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJN9WRF/build/release/bin/kudu
/tmp/dist-test-taskJN9WRF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790257629041645-13911-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790257629041645-13911-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790257629041645-13911-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790257629041645-13911-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.149.254:45043
--webserver_interface=127.13.149.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.254.keytab
--principal=kudu/127.13.149.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.149.212:45653
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.149.254:45043
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790257629041645-13911-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790257629041645-13911-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790257629041645-13911-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 13:48:13.646944 22177 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 13:48:13.647200 22177 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 13:48:13.647225 22177 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 13:48:13.648586 22177 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 13:48:13.648661 22177 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 13:48:13.648675 22177 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 13:48:13.648687 22177 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 13:48:13.648697 22177 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 13:48:13.650496 22177 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.149.212:45653
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790257629041645-13911-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790257629041645-13911-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.149.254:45043
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.254.keytab
--principal=kudu/127.13.149.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.149.254:45043
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790257629041645-13911-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.149.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22177
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790257629041645-13911-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 12:29:37 UTC on ccycloud.quasar-iqzbxn.root.comops.site
build id 4670106
I20260924 13:48:13.650808 22177 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 13:48:13.651052 22177 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 13:48:13.656447 22183 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 13:48:13.656375 22182 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 13:48:13.656466 22177 server_base.cc:1034] running on GCE node
W20260924 13:48:13.656379 22185 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 13:48:13.656970 22177 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 13:48:13.657347 22177 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 13:48:13.658538 22177 hybrid_clock.cc:648] HybridClock initialized: now 1790257693658523 us; error 43 us; skew 500 ppm
Sep 24 13:48:13 dist-test-slave-sjl2 krb5kdc[22164](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257693, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.149.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 13:48:13.666404 22177 init.cc:377] Logged in from keytab as kudu/127.13.149.254@KRBTEST.COM (short username kudu)
I20260924 13:48:13.666898 22177 webserver.cc:460] Webserver started at http://127.13.149.254:42957/ using document root <none> and password file <none>
I20260924 13:48:13.667117 22177 fs_manager.cc:359] Metadata directory not provided
I20260924 13:48:13.667168 22177 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 13:48:13.667297 22177 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 13:48:13.668434 22177 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790257629041645-13911-0/minicluster-data/master-0/data/instance:
uuid: "94bf763b8fce4015898b8ee17c4faf3e"
format_stamp: "Formatted at 2026-09-24 13:48:13 on dist-test-slave-sjl2"
server_key: "2044300065c819ff0fcf332e98b6e733"
server_key_iv: "3afb9c0a4cb5a6e5c46b339df4138504"
server_key_version: "encryptionkey@0"
I20260924 13:48:13.668771 22177 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790257629041645-13911-0/minicluster-data/master-0/wal/instance:
uuid: "94bf763b8fce4015898b8ee17c4faf3e"
format_stamp: "Formatted at 2026-09-24 13:48:13 on dist-test-slave-sjl2"
server_key: "2044300065c819ff0fcf332e98b6e733"
server_key_iv: "3afb9c0a4cb5a6e5c46b339df4138504"
server_key_version: "encryptionkey@0"
I20260924 13:48:13.670181 22177 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260924 13:48:13.670900 22192 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 13:48:13.671080 22177 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260924 13:48:13.671177 22177 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790257629041645-13911-0/minicluster-data/master-0/data,/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790257629041645-13911-0/minicluster-data/master-0/wal
uuid: "94bf763b8fce4015898b8ee17c4faf3e"
format_stamp: "Formatted at 2026-09-24 13:48:13 on dist-test-slave-sjl2"
server_key: "2044300065c819ff0fcf332e98b6e733"
server_key_iv: "3afb9c0a4cb5a6e5c46b339df4138504"
server_key_version: "encryptionkey@0"
I20260924 13:48:13.671243 22177 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790257629041645-13911-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790257629041645-13911-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790257629041645-13911-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 13:48:13.683820 22177 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 13:48:13.686054 22177 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 13:48:13.686204 22177 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 13:48:13.690090 22177 rpc_server.cc:307] RPC server started. Bound to: 127.13.149.254:45043
I20260924 13:48:13.690131 22244 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.149.254:45043 every 8 connection(s)
I20260924 13:48:13.690466 22177 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790257629041645-13911-0/minicluster-data/master-0/data/info.pb
I20260924 13:48:13.691351 22245 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 13:48:13.693765 22245 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 94bf763b8fce4015898b8ee17c4faf3e: Bootstrap starting.
I20260924 13:48:13.694355 22245 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 94bf763b8fce4015898b8ee17c4faf3e: Neither blocks nor log segments found. Creating new log.
I20260924 13:48:13.694615 22245 log.cc:824] T 00000000000000000000000000000000 P 94bf763b8fce4015898b8ee17c4faf3e: Log is configured to *not* fsync() on all Append() calls
I20260924 13:48:13.695343 22245 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 94bf763b8fce4015898b8ee17c4faf3e: No bootstrap required, opened a new log
I20260924 13:48:13.695883 13911 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJN9WRF/build/release/bin/kudu as pid 22177
I20260924 13:48:13.696007 13911 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790257629041645-13911-0/minicluster-data/master-0/wal/instance
I20260924 13:48:13.696177 13911 external_mini_cluster.cc:1428] Setting key 0a6e1a2a4fe233d525e51904b29ccd19
I20260924 13:48:13.697278 22245 raft_consensus.cc:348] T 00000000000000000000000000000000 P 94bf763b8fce4015898b8ee17c4faf3e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "94bf763b8fce4015898b8ee17c4faf3e" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 45043 } }
I20260924 13:48:13.697438 22245 raft_consensus.cc:374] T 00000000000000000000000000000000 P 94bf763b8fce4015898b8ee17c4faf3e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 13:48:13.697474 22245 raft_consensus.cc:729] T 00000000000000000000000000000000 P 94bf763b8fce4015898b8ee17c4faf3e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 94bf763b8fce4015898b8ee17c4faf3e, State: Initialized, Role: FOLLOWER
I20260924 13:48:13.697638 22245 consensus_queue.cc:260] T 00000000000000000000000000000000 P 94bf763b8fce4015898b8ee17c4faf3e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "94bf763b8fce4015898b8ee17c4faf3e" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 45043 } }
I20260924 13:48:13.697723 22245 raft_consensus.cc:388] T 00000000000000000000000000000000 P 94bf763b8fce4015898b8ee17c4faf3e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 13:48:13.697754 22245 raft_consensus.cc:482] T 00000000000000000000000000000000 P 94bf763b8fce4015898b8ee17c4faf3e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 13:48:13.697801 22245 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 94bf763b8fce4015898b8ee17c4faf3e [term 0 FOLLOWER]: Advancing to term 1
I20260924 13:48:13.698777 22245 raft_consensus.cc:504] T 00000000000000000000000000000000 P 94bf763b8fce4015898b8ee17c4faf3e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "94bf763b8fce4015898b8ee17c4faf3e" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 45043 } }
I20260924 13:48:13.698874 22245 leader_election.cc:302] T 00000000000000000000000000000000 P 94bf763b8fce4015898b8ee17c4faf3e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 94bf763b8fce4015898b8ee17c4faf3e; no voters:
I20260924 13:48:13.699044 22245 leader_election.cc:288] T 00000000000000000000000000000000 P 94bf763b8fce4015898b8ee17c4faf3e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 13:48:13.699070 22250 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 94bf763b8fce4015898b8ee17c4faf3e [term 1 FOLLOWER]: Leader election won for term 1
I20260924 13:48:13.699170 22250 raft_consensus.cc:686] T 00000000000000000000000000000000 P 94bf763b8fce4015898b8ee17c4faf3e [term 1 LEADER]: Becoming Leader. State: Replica: 94bf763b8fce4015898b8ee17c4faf3e, State: Running, Role: LEADER
I20260924 13:48:13.699256 22250 consensus_queue.cc:237] T 00000000000000000000000000000000 P 94bf763b8fce4015898b8ee17c4faf3e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "94bf763b8fce4015898b8ee17c4faf3e" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 45043 } }
I20260924 13:48:13.699361 22245 sys_catalog.cc:564] T 00000000000000000000000000000000 P 94bf763b8fce4015898b8ee17c4faf3e [sys.catalog]: configured and running, proceeding with master startup.
I20260924 13:48:13.699648 22250 sys_catalog.cc:455] T 00000000000000000000000000000000 P 94bf763b8fce4015898b8ee17c4faf3e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "94bf763b8fce4015898b8ee17c4faf3e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "94bf763b8fce4015898b8ee17c4faf3e" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 45043 } } }
I20260924 13:48:13.699718 22250 sys_catalog.cc:458] T 00000000000000000000000000000000 P 94bf763b8fce4015898b8ee17c4faf3e [sys.catalog]: This master's current role is: LEADER
I20260924 13:48:13.699827 22250 sys_catalog.cc:455] T 00000000000000000000000000000000 P 94bf763b8fce4015898b8ee17c4faf3e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 94bf763b8fce4015898b8ee17c4faf3e. Latest consensus state: current_term: 1 leader_uuid: "94bf763b8fce4015898b8ee17c4faf3e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "94bf763b8fce4015898b8ee17c4faf3e" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 45043 } } }
I20260924 13:48:13.699882 22250 sys_catalog.cc:458] T 00000000000000000000000000000000 P 94bf763b8fce4015898b8ee17c4faf3e [sys.catalog]: This master's current role is: LEADER
I20260924 13:48:13.700140 22255 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 13:48:13.700690 22255 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 13:48:13.702381 22255 catalog_manager.cc:1347] Generated new cluster ID: fb9fd0abb2144a448e7a4acccf23e91f
I20260924 13:48:13.702438 22255 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 24 13:48:13 dist-test-slave-sjl2 krb5kdc[22164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257693, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.149.254@KRBTEST.COM
I20260924 13:48:13.707727 22249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:48:13.696707 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57916 (local address 127.13.149.254:45043)
0924 13:48:13.696884 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0924 13:48:13.696892 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0924 13:48:13.696939 (+ 47us) server_negotiation.cc:407] Connection header received
0924 13:48:13.697240 (+ 301us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:48:13.697245 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:48:13.697291 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:48:13.697404 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:48:13.698149 (+ 745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:48:13.699260 (+ 1111us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:48:13.700238 (+ 978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:48:13.700831 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:48:13.700974 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:48:13.703877 (+ 2903us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:48:13.703881 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:48:13.703881 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 13:48:13.703894 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:48:13.705830 (+ 1936us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:48:13.706195 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:48:13.706198 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:48:13.706198 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:48:13.706321 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:48:13.706499 (+ 178us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:48:13.706499 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:48:13.706500 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:48:13.706805 (+ 305us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:48:13.706931 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0924 13:48:13.707344 (+ 413us) server_negotiation.cc:299] Negotiation successful
0924 13:48:13.707498 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125,"thread_start_us":70,"threads_started":1}
I20260924 13:48:13.727135 22255 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 13:48:13.727763 22255 catalog_manager.cc:1504] Loading token signing keys...
I20260924 13:48:13.732141 22255 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 94bf763b8fce4015898b8ee17c4faf3e: Generated new TSK 0
I20260924 13:48:13.732347 22255 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.13.149.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.149.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.149.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.193.keytab.
Entry for principal kudu/127.13.149.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.193.keytab.
I20260924 13:48:13.761526 13911 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJN9WRF/build/release/bin/kudu
/tmp/dist-test-taskJN9WRF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790257629041645-13911-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790257629041645-13911-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790257629041645-13911-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790257629041645-13911-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.149.193:0
--local_ip_for_outbound_sockets=127.13.149.193
--webserver_interface=127.13.149.193
--webserver_port=0
--tserver_master_addrs=127.13.149.254:45043
--keytab_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.193.keytab
--principal=kudu/127.13.149.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.149.212:45653
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790257629041645-13911-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790257629041645-13911-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790257629041645-13911-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 13:48:13.844568 22271 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 13:48:13.844798 22271 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 13:48:13.844831 22271 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 13:48:13.846124 22271 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 13:48:13.846179 22271 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 13:48:13.846220 22271 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.149.193
I20260924 13:48:13.847775 22271 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.149.212:45653
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790257629041645-13911-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790257629041645-13911-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.193.keytab
--principal=kudu/127.13.149.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.149.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790257629041645-13911-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.149.193
--webserver_port=0
--tserver_master_addrs=127.13.149.254:45043
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22271
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.149.193
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790257629041645-13911-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 12:29:37 UTC on ccycloud.quasar-iqzbxn.root.comops.site
build id 4670106
I20260924 13:48:13.848052 22271 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 13:48:13.848280 22271 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260924 13:48:13.853328 22271 server_base.cc:1034] running on GCE node
W20260924 13:48:13.853252 22277 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 13:48:13.853322 22276 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 13:48:13.853258 22279 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 13:48:13.853717 22271 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 13:48:13.853984 22271 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 13:48:13.855144 22271 hybrid_clock.cc:648] HybridClock initialized: now 1790257693855123 us; error 35 us; skew 500 ppm
Sep 24 13:48:13 dist-test-slave-sjl2 krb5kdc[22164](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257693, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.149.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 13:48:13.857715 22271 init.cc:377] Logged in from keytab as kudu/127.13.149.193@KRBTEST.COM (short username kudu)
I20260924 13:48:13.858194 22271 webserver.cc:460] Webserver started at http://127.13.149.193:42325/ using document root <none> and password file <none>
I20260924 13:48:13.858417 22271 fs_manager.cc:359] Metadata directory not provided
I20260924 13:48:13.858469 22271 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 13:48:13.858595 22271 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 13:48:13.859650 22271 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790257629041645-13911-0/minicluster-data/ts-0/data/instance:
uuid: "2842fbd855834006a6b4ca8c5a11db24"
format_stamp: "Formatted at 2026-09-24 13:48:13 on dist-test-slave-sjl2"
server_key: "8a6783ebdb24ebf935507b8db2c3b833"
server_key_iv: "fe0fc849df5ce01c094f453374e11398"
server_key_version: "encryptionkey@0"
I20260924 13:48:13.859979 22271 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790257629041645-13911-0/minicluster-data/ts-0/wal/instance:
uuid: "2842fbd855834006a6b4ca8c5a11db24"
format_stamp: "Formatted at 2026-09-24 13:48:13 on dist-test-slave-sjl2"
server_key: "8a6783ebdb24ebf935507b8db2c3b833"
server_key_iv: "fe0fc849df5ce01c094f453374e11398"
server_key_version: "encryptionkey@0"
I20260924 13:48:13.861375 22271 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260924 13:48:13.862128 22286 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 13:48:13.862277 22271 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260924 13:48:13.862360 22271 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790257629041645-13911-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790257629041645-13911-0/minicluster-data/ts-0/wal
uuid: "2842fbd855834006a6b4ca8c5a11db24"
format_stamp: "Formatted at 2026-09-24 13:48:13 on dist-test-slave-sjl2"
server_key: "8a6783ebdb24ebf935507b8db2c3b833"
server_key_iv: "fe0fc849df5ce01c094f453374e11398"
server_key_version: "encryptionkey@0"
I20260924 13:48:13.862426 22271 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790257629041645-13911-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790257629041645-13911-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790257629041645-13911-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 13:48:13.871013 22271 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 13:48:13.873097 22271 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 13:48:13.873242 22271 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 13:48:13.873456 22271 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 13:48:13.873744 22271 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 13:48:13.873776 22271 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 13:48:13.873806 22271 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 13:48:13.873829 22271 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 13:48:13.879523 22271 rpc_server.cc:307] RPC server started. Bound to: 127.13.149.193:41099
I20260924 13:48:13.879585 22399 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.149.193:41099 every 8 connection(s)
I20260924 13:48:13.879890 22271 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790257629041645-13911-0/minicluster-data/ts-0/data/info.pb
Sep 24 13:48:13 dist-test-slave-sjl2 krb5kdc[22164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257693, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.149.193@KRBTEST.COM for kudu/127.13.149.254@KRBTEST.COM
I20260924 13:48:13.886761 13911 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJN9WRF/build/release/bin/kudu as pid 22271
I20260924 13:48:13.886860 13911 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790257629041645-13911-0/minicluster-data/ts-0/wal/instance
I20260924 13:48:13.887048 13911 external_mini_cluster.cc:1428] Setting key a04da9c1f10ec1d31f7a51a798e99219
I20260924 13:48:13.889446 22249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:48:13.880632 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.149.193:34931 (local address 127.13.149.254:45043)
0924 13:48:13.880690 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0924 13:48:13.880692 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:48:13.880895 (+ 203us) server_negotiation.cc:407] Connection header received
0924 13:48:13.881469 (+ 574us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:48:13.881471 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:48:13.881486 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:48:13.881558 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:48:13.882509 (+ 951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:48:13.883039 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:48:13.884037 (+ 998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:48:13.884240 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:48:13.884293 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:48:13.886465 (+ 2172us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:48:13.886467 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:48:13.886467 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 13:48:13.886479 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:48:13.887941 (+ 1462us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:48:13.888413 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:48:13.888414 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:48:13.888415 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:48:13.888474 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:48:13.888721 (+ 247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:48:13.888722 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:48:13.888722 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:48:13.888828 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:48:13.888871 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0924 13:48:13.889235 (+ 364us) server_negotiation.cc:299] Negotiation successful
0924 13:48:13.889345 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260924 13:48:13.889555 22402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:48:13.880610 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:45043 (local address 127.13.149.193:34931)
0924 13:48:13.880771 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0924 13:48:13.880815 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0924 13:48:13.881348 (+ 533us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:48:13.881616 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:48:13.881620 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:48:13.882193 (+ 573us) client_negotiation.cc:189] Negotiated authn=SASL
0924 13:48:13.882394 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:48:13.882398 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:48:13.883146 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:48:13.883147 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:48:13.883966 (+ 819us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:48:13.883968 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:48:13.884303 (+ 335us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:48:13.884304 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:48:13.884410 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:48:13.884880 (+ 470us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 13:48:13.884889 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0924 13:48:13.886356 (+ 1467us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 13:48:13.888033 (+ 1677us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 13:48:13.888044 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 13:48:13.888047 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0924 13:48:13.888339 (+ 292us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 13:48:13.888526 (+ 187us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 13:48:13.888527 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 13:48:13.888527 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 13:48:13.888665 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 13:48:13.888874 (+ 209us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 13:48:13.888878 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 13:48:13.889078 (+ 200us) client_negotiation.cc:769] Sending connection context
0924 13:48:13.889202 (+ 124us) client_negotiation.cc:240] Negotiation successful
0924 13:48:13.889362 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":75,"threads_started":1}
I20260924 13:48:13.890030 22400 heartbeater.cc:347] Connected to a master server at 127.13.149.254:45043
I20260924 13:48:13.890144 22400 heartbeater.cc:464] Registering TS with master...
I20260924 13:48:13.890314 22400 heartbeater.cc:511] Master 127.13.149.254:45043 requested a full tablet report, sending...
I20260924 13:48:13.890794 22209 ts_manager.cc:194] Registered new tserver with Master: 2842fbd855834006a6b4ca8c5a11db24 (127.13.149.193:41099)
I20260924 13:48:13.891393 22209 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.149.193@KRBTEST.COM'} at 127.13.149.193:34931
WARNING: no policy specified for kudu/127.13.149.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.149.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.149.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.194.keytab.
Entry for principal kudu/127.13.149.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.194.keytab.
I20260924 13:48:13.913853 13911 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJN9WRF/build/release/bin/kudu
/tmp/dist-test-taskJN9WRF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790257629041645-13911-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790257629041645-13911-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790257629041645-13911-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790257629041645-13911-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.149.194:0
--local_ip_for_outbound_sockets=127.13.149.194
--webserver_interface=127.13.149.194
--webserver_port=0
--tserver_master_addrs=127.13.149.254:45043
--keytab_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.194.keytab
--principal=kudu/127.13.149.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.149.212:45653
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790257629041645-13911-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790257629041645-13911-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790257629041645-13911-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 13:48:13.999101 22405 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 13:48:13.999336 22405 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 13:48:13.999373 22405 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 13:48:14.000773 22405 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 13:48:14.000833 22405 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 13:48:14.000875 22405 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.149.194
I20260924 13:48:14.002542 22405 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.149.212:45653
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790257629041645-13911-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790257629041645-13911-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.194.keytab
--principal=kudu/127.13.149.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.149.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790257629041645-13911-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.149.194
--webserver_port=0
--tserver_master_addrs=127.13.149.254:45043
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22405
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.149.194
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790257629041645-13911-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 12:29:37 UTC on ccycloud.quasar-iqzbxn.root.comops.site
build id 4670106
I20260924 13:48:14.002818 22405 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 13:48:14.003057 22405 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 13:48:14.008486 22413 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 13:48:14.008486 22411 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 13:48:14.008610 22405 server_base.cc:1034] running on GCE node
W20260924 13:48:14.008706 22410 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 13:48:14.009042 22405 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 13:48:14.009284 22405 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 13:48:14.010452 22405 hybrid_clock.cc:648] HybridClock initialized: now 1790257694010435 us; error 33 us; skew 500 ppm
Sep 24 13:48:14 dist-test-slave-sjl2 krb5kdc[22164](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257694, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.149.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 13:48:14.013168 22405 init.cc:377] Logged in from keytab as kudu/127.13.149.194@KRBTEST.COM (short username kudu)
I20260924 13:48:14.013646 22405 webserver.cc:460] Webserver started at http://127.13.149.194:42049/ using document root <none> and password file <none>
I20260924 13:48:14.013888 22405 fs_manager.cc:359] Metadata directory not provided
I20260924 13:48:14.013937 22405 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 13:48:14.014041 22405 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 13:48:14.015188 22405 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790257629041645-13911-0/minicluster-data/ts-1/data/instance:
uuid: "d0687009a62c4ec7a045f6856ef0be30"
format_stamp: "Formatted at 2026-09-24 13:48:14 on dist-test-slave-sjl2"
server_key: "913f4420175e9fe624c4863c06b58e59"
server_key_iv: "3b2a90f909090b9443a1c8a9213a7f0a"
server_key_version: "encryptionkey@0"
I20260924 13:48:14.015539 22405 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790257629041645-13911-0/minicluster-data/ts-1/wal/instance:
uuid: "d0687009a62c4ec7a045f6856ef0be30"
format_stamp: "Formatted at 2026-09-24 13:48:14 on dist-test-slave-sjl2"
server_key: "913f4420175e9fe624c4863c06b58e59"
server_key_iv: "3b2a90f909090b9443a1c8a9213a7f0a"
server_key_version: "encryptionkey@0"
I20260924 13:48:14.017063 22405 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260924 13:48:14.018056 22420 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 13:48:14.018309 22405 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260924 13:48:14.018415 22405 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790257629041645-13911-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790257629041645-13911-0/minicluster-data/ts-1/wal
uuid: "d0687009a62c4ec7a045f6856ef0be30"
format_stamp: "Formatted at 2026-09-24 13:48:14 on dist-test-slave-sjl2"
server_key: "913f4420175e9fe624c4863c06b58e59"
server_key_iv: "3b2a90f909090b9443a1c8a9213a7f0a"
server_key_version: "encryptionkey@0"
I20260924 13:48:14.018501 22405 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790257629041645-13911-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790257629041645-13911-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790257629041645-13911-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 13:48:14.029949 22405 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 13:48:14.032749 22405 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 13:48:14.032991 22405 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 13:48:14.033252 22405 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 13:48:14.033574 22405 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 13:48:14.033609 22405 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 13:48:14.033641 22405 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 13:48:14.033665 22405 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 13:48:14.039939 22405 rpc_server.cc:307] RPC server started. Bound to: 127.13.149.194:42801
I20260924 13:48:14.040051 22533 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.149.194:42801 every 8 connection(s)
I20260924 13:48:14.040324 22405 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790257629041645-13911-0/minicluster-data/ts-1/data/info.pb
Sep 24 13:48:14 dist-test-slave-sjl2 krb5kdc[22164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257694, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.149.194@KRBTEST.COM for kudu/127.13.149.254@KRBTEST.COM
I20260924 13:48:14.049651 13911 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJN9WRF/build/release/bin/kudu as pid 22405
I20260924 13:48:14.049777 13911 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790257629041645-13911-0/minicluster-data/ts-1/wal/instance
I20260924 13:48:14.050002 13911 external_mini_cluster.cc:1428] Setting key bb156e0a3d74b5cc0eeeac162c9fa473
I20260924 13:48:14.050203 22249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:48:14.041117 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.149.194:50089 (local address 127.13.149.254:45043)
0924 13:48:14.041170 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0924 13:48:14.041172 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:48:14.041399 (+ 227us) server_negotiation.cc:407] Connection header received
0924 13:48:14.041939 (+ 540us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:48:14.041940 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:48:14.041954 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:48:14.042003 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:48:14.043055 (+ 1052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:48:14.043578 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:48:14.044662 (+ 1084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:48:14.044873 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:48:14.044961 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:48:14.047286 (+ 2325us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:48:14.047288 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:48:14.047288 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 13:48:14.047296 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:48:14.048600 (+ 1304us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:48:14.049072 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:48:14.049073 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:48:14.049073 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:48:14.049128 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:48:14.049413 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:48:14.049414 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:48:14.049415 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:48:14.049549 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:48:14.049626 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0924 13:48:14.049995 (+ 369us) server_negotiation.cc:299] Negotiation successful
0924 13:48:14.050108 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260924 13:48:14.050451 22536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:48:14.041084 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:45043 (local address 127.13.149.194:50089)
0924 13:48:14.041264 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0924 13:48:14.041304 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0924 13:48:14.041825 (+ 521us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:48:14.042091 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:48:14.042096 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:48:14.042742 (+ 646us) client_negotiation.cc:189] Negotiated authn=SASL
0924 13:48:14.042949 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:48:14.042954 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:48:14.043670 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:48:14.043672 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:48:14.044561 (+ 889us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:48:14.044564 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:48:14.044979 (+ 415us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:48:14.044980 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:48:14.045127 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:48:14.045665 (+ 538us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 13:48:14.045674 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0924 13:48:14.047160 (+ 1486us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 13:48:14.048699 (+ 1539us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 13:48:14.048708 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 13:48:14.048710 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 13:48:14.049007 (+ 297us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 13:48:14.049204 (+ 197us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 13:48:14.049205 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 13:48:14.049205 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 13:48:14.049326 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 13:48:14.049641 (+ 315us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 13:48:14.049649 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 13:48:14.049851 (+ 202us) client_negotiation.cc:769] Sending connection context
0924 13:48:14.049976 (+ 125us) client_negotiation.cc:240] Negotiation successful
0924 13:48:14.050180 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":87,"threads_started":1}
I20260924 13:48:14.050943 22534 heartbeater.cc:347] Connected to a master server at 127.13.149.254:45043
I20260924 13:48:14.051052 22534 heartbeater.cc:464] Registering TS with master...
I20260924 13:48:14.051232 22534 heartbeater.cc:511] Master 127.13.149.254:45043 requested a full tablet report, sending...
I20260924 13:48:14.051647 22209 ts_manager.cc:194] Registered new tserver with Master: d0687009a62c4ec7a045f6856ef0be30 (127.13.149.194:42801)
I20260924 13:48:14.052094 22209 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.149.194@KRBTEST.COM'} at 127.13.149.194:50089
WARNING: no policy specified for kudu/127.13.149.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.149.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.149.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.195.keytab.
Entry for principal kudu/127.13.149.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.195.keytab.
I20260924 13:48:14.076514 13911 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJN9WRF/build/release/bin/kudu
/tmp/dist-test-taskJN9WRF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790257629041645-13911-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790257629041645-13911-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790257629041645-13911-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790257629041645-13911-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.149.195:0
--local_ip_for_outbound_sockets=127.13.149.195
--webserver_interface=127.13.149.195
--webserver_port=0
--tserver_master_addrs=127.13.149.254:45043
--keytab_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.195.keytab
--principal=kudu/127.13.149.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.149.212:45653
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790257629041645-13911-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790257629041645-13911-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790257629041645-13911-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 13:48:14.158012 22539 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 13:48:14.158246 22539 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 13:48:14.158273 22539 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 13:48:14.159516 22539 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 13:48:14.159560 22539 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 13:48:14.159590 22539 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.149.195
I20260924 13:48:14.161209 22539 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.149.212:45653
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790257629041645-13911-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790257629041645-13911-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.195.keytab
--principal=kudu/127.13.149.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.149.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790257629041645-13911-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.149.195
--webserver_port=0
--tserver_master_addrs=127.13.149.254:45043
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22539
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.149.195
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790257629041645-13911-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 12:29:37 UTC on ccycloud.quasar-iqzbxn.root.comops.site
build id 4670106
I20260924 13:48:14.161453 22539 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 13:48:14.161655 22539 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 13:48:14.166770 22544 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 13:48:14.166888 22539 server_base.cc:1034] running on GCE node
W20260924 13:48:14.166999 22545 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 13:48:14.166770 22547 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 13:48:14.167249 22539 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 13:48:14.167518 22539 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 13:48:14.168670 22539 hybrid_clock.cc:648] HybridClock initialized: now 1790257694168651 us; error 35 us; skew 500 ppm
Sep 24 13:48:14 dist-test-slave-sjl2 krb5kdc[22164](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257694, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.149.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 13:48:14.171101 22539 init.cc:377] Logged in from keytab as kudu/127.13.149.195@KRBTEST.COM (short username kudu)
I20260924 13:48:14.171548 22539 webserver.cc:460] Webserver started at http://127.13.149.195:39967/ using document root <none> and password file <none>
I20260924 13:48:14.171744 22539 fs_manager.cc:359] Metadata directory not provided
I20260924 13:48:14.171787 22539 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 13:48:14.171912 22539 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 13:48:14.173007 22539 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790257629041645-13911-0/minicluster-data/ts-2/data/instance:
uuid: "d3f5f2aaae334a7ebf34d1872226b5a5"
format_stamp: "Formatted at 2026-09-24 13:48:14 on dist-test-slave-sjl2"
server_key: "9b07c7f9152f1097a2c49c1a4e67447d"
server_key_iv: "7ae450f58e8979bd161a4e2ff19140da"
server_key_version: "encryptionkey@0"
I20260924 13:48:14.173380 22539 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790257629041645-13911-0/minicluster-data/ts-2/wal/instance:
uuid: "d3f5f2aaae334a7ebf34d1872226b5a5"
format_stamp: "Formatted at 2026-09-24 13:48:14 on dist-test-slave-sjl2"
server_key: "9b07c7f9152f1097a2c49c1a4e67447d"
server_key_iv: "7ae450f58e8979bd161a4e2ff19140da"
server_key_version: "encryptionkey@0"
I20260924 13:48:14.174814 22539 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260924 13:48:14.175693 22554 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 13:48:14.175895 22539 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260924 13:48:14.176003 22539 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790257629041645-13911-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790257629041645-13911-0/minicluster-data/ts-2/wal
uuid: "d3f5f2aaae334a7ebf34d1872226b5a5"
format_stamp: "Formatted at 2026-09-24 13:48:14 on dist-test-slave-sjl2"
server_key: "9b07c7f9152f1097a2c49c1a4e67447d"
server_key_iv: "7ae450f58e8979bd161a4e2ff19140da"
server_key_version: "encryptionkey@0"
I20260924 13:48:14.176080 22539 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790257629041645-13911-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790257629041645-13911-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790257629041645-13911-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 13:48:14.187551 22539 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 13:48:14.189745 22539 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 13:48:14.189888 22539 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 13:48:14.190107 22539 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 13:48:14.190402 22539 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 13:48:14.190433 22539 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 13:48:14.190462 22539 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 13:48:14.190483 22539 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 13:48:14.196019 22539 rpc_server.cc:307] RPC server started. Bound to: 127.13.149.195:42947
I20260924 13:48:14.196064 22667 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.149.195:42947 every 8 connection(s)
I20260924 13:48:14.196375 22539 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790257629041645-13911-0/minicluster-data/ts-2/data/info.pb
I20260924 13:48:14.201483 13911 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJN9WRF/build/release/bin/kudu as pid 22539
I20260924 13:48:14.201601 13911 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790257629041645-13911-0/minicluster-data/ts-2/wal/instance
I20260924 13:48:14.201764 13911 external_mini_cluster.cc:1428] Setting key b12dedd33f053abd88eeb630644d6e57
Sep 24 13:48:14 dist-test-slave-sjl2 krb5kdc[22164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257694, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.149.195@KRBTEST.COM for kudu/127.13.149.254@KRBTEST.COM
I20260924 13:48:14.205828 22249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:48:14.197110 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.149.195:59795 (local address 127.13.149.254:45043)
0924 13:48:14.197159 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0924 13:48:14.197161 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:48:14.197380 (+ 219us) server_negotiation.cc:407] Connection header received
0924 13:48:14.197846 (+ 466us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:48:14.197847 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:48:14.197857 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:48:14.197909 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:48:14.198867 (+ 958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:48:14.199440 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:48:14.200614 (+ 1174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:48:14.200813 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:48:14.200871 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:48:14.203209 (+ 2338us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:48:14.203211 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:48:14.203211 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 13:48:14.203220 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:48:14.204408 (+ 1188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:48:14.204816 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:48:14.204818 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:48:14.204818 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:48:14.204886 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:48:14.205138 (+ 252us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:48:14.205139 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:48:14.205139 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:48:14.205272 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:48:14.205348 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0924 13:48:14.205635 (+ 287us) server_negotiation.cc:299] Negotiation successful
0924 13:48:14.205729 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260924 13:48:14.205997 22670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:48:14.197100 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:45043 (local address 127.13.149.195:59795)
0924 13:48:14.197262 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0924 13:48:14.197301 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0924 13:48:14.197768 (+ 467us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:48:14.197962 (+ 194us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:48:14.197965 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:48:14.198575 (+ 610us) client_negotiation.cc:189] Negotiated authn=SASL
0924 13:48:14.198780 (+ 205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:48:14.198786 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:48:14.199557 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:48:14.199559 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:48:14.200472 (+ 913us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:48:14.200476 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:48:14.200891 (+ 415us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:48:14.200892 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:48:14.201063 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:48:14.201636 (+ 573us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 13:48:14.201647 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0924 13:48:14.203110 (+ 1463us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 13:48:14.204494 (+ 1384us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 13:48:14.204503 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 13:48:14.204505 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 13:48:14.204741 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 13:48:14.204963 (+ 222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 13:48:14.204963 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 13:48:14.204964 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 13:48:14.205077 (+ 113us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 13:48:14.205331 (+ 254us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 13:48:14.205337 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 13:48:14.205486 (+ 149us) client_negotiation.cc:769] Sending connection context
0924 13:48:14.205591 (+ 105us) client_negotiation.cc:240] Negotiation successful
0924 13:48:14.205758 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":86,"threads_started":1}
I20260924 13:48:14.206461 22668 heartbeater.cc:347] Connected to a master server at 127.13.149.254:45043
I20260924 13:48:14.206555 22668 heartbeater.cc:464] Registering TS with master...
I20260924 13:48:14.206745 22668 heartbeater.cc:511] Master 127.13.149.254:45043 requested a full tablet report, sending...
I20260924 13:48:14.207134 22209 ts_manager.cc:194] Registered new tserver with Master: d3f5f2aaae334a7ebf34d1872226b5a5 (127.13.149.195:42947)
I20260924 13:48:14.207520 22209 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.149.195@KRBTEST.COM'} at 127.13.149.195:59795
I20260924 13:48:14.213719 13911 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.13.149.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.13.149.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.13.149.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790257629041645-13911-0/krb5kdc/oryx_127.13.149.193.keytab.
Entry for principal oryx/127.13.149.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790257629041645-13911-0/krb5kdc/oryx_127.13.149.193.keytab.
WARNING: no policy specified for oryx/127.13.149.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.13.149.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.13.149.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790257629041645-13911-0/krb5kdc/oryx_127.13.149.194.keytab.
Entry for principal oryx/127.13.149.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790257629041645-13911-0/krb5kdc/oryx_127.13.149.194.keytab.
WARNING: no policy specified for oryx/127.13.149.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.13.149.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.13.149.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790257629041645-13911-0/krb5kdc/oryx_127.13.149.195.keytab.
Entry for principal oryx/127.13.149.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790257629041645-13911-0/krb5kdc/oryx_127.13.149.195.keytab.
I20260924 13:48:14.289947 13911 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJN9WRF/build/release/bin/kudu with pid 22271
I20260924 13:48:14.294062 13911 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJN9WRF/build/release/bin/kudu with pid 22405
I20260924 13:48:14.297993 13911 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJN9WRF/build/release/bin/kudu with pid 22539
I20260924 13:48:14.301920 13911 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJN9WRF/build/release/bin/kudu with pid 22177
I20260924 13:48:14.306651 13911 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJN9WRF/build/release/bin/kudu
/tmp/dist-test-taskJN9WRF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790257629041645-13911-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790257629041645-13911-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790257629041645-13911-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790257629041645-13911-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.149.254:45043
--webserver_interface=127.13.149.254
--webserver_port=42957
--keytab_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.254.keytab
--principal=kudu/127.13.149.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.149.212:45653
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.149.254:45043
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790257629041645-13911-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790257629041645-13911-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790257629041645-13911-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 13:48:14.390873 22677 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 13:48:14.391108 22677 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 13:48:14.391132 22677 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 13:48:14.392366 22677 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 13:48:14.392431 22677 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 13:48:14.392444 22677 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 13:48:14.392457 22677 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 13:48:14.392480 22677 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 13:48:14.394454 22677 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.149.212:45653
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790257629041645-13911-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790257629041645-13911-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.149.254:45043
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.254.keytab
--principal=kudu/127.13.149.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.149.254:45043
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790257629041645-13911-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.149.254
--webserver_port=42957
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22677
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790257629041645-13911-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 12:29:37 UTC on ccycloud.quasar-iqzbxn.root.comops.site
build id 4670106
I20260924 13:48:14.394840 22677 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 13:48:14.395102 22677 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 13:48:14.401217 22685 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 13:48:14.401254 22682 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 13:48:14.401230 22683 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 13:48:14.401582 22677 server_base.cc:1034] running on GCE node
I20260924 13:48:14.401856 22677 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 13:48:14.402194 22677 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 13:48:14.403395 22677 hybrid_clock.cc:648] HybridClock initialized: now 1790257694403360 us; error 71 us; skew 500 ppm
Sep 24 13:48:14 dist-test-slave-sjl2 krb5kdc[22164](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257694, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.149.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 13:48:14.406189 22677 init.cc:377] Logged in from keytab as kudu/127.13.149.254@KRBTEST.COM (short username kudu)
I20260924 13:48:14.406677 22677 webserver.cc:460] Webserver started at http://127.13.149.254:42957/ using document root <none> and password file <none>
I20260924 13:48:14.406891 22677 fs_manager.cc:359] Metadata directory not provided
I20260924 13:48:14.406955 22677 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 13:48:14.408394 22677 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260924 13:48:14.409264 22692 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 13:48:14.409600 22677 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260924 13:48:14.409716 22677 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790257629041645-13911-0/minicluster-data/master-0/data,/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790257629041645-13911-0/minicluster-data/master-0/wal
uuid: "94bf763b8fce4015898b8ee17c4faf3e"
format_stamp: "Formatted at 2026-09-24 13:48:13 on dist-test-slave-sjl2"
server_key: "2044300065c819ff0fcf332e98b6e733"
server_key_iv: "3afb9c0a4cb5a6e5c46b339df4138504"
server_key_version: "encryptionkey@0"
I20260924 13:48:14.410100 22677 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790257629041645-13911-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790257629041645-13911-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790257629041645-13911-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 13:48:14.417449 22677 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 13:48:14.419507 22677 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 13:48:14.419651 22677 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 13:48:14.423568 22677 rpc_server.cc:307] RPC server started. Bound to: 127.13.149.254:45043
I20260924 13:48:14.423817 22744 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.149.254:45043 every 8 connection(s)
I20260924 13:48:14.424098 22677 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790257629041645-13911-0/minicluster-data/master-0/data/info.pb
I20260924 13:48:14.425683 22745 sys_catalog.cc:263] Verifying existing consensus state
I20260924 13:48:14.426311 22745 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 94bf763b8fce4015898b8ee17c4faf3e: Bootstrap starting.
I20260924 13:48:14.431790 13911 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJN9WRF/build/release/bin/kudu as pid 22677
I20260924 13:48:14.432108 13911 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJN9WRF/build/release/bin/kudu
/tmp/dist-test-taskJN9WRF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790257629041645-13911-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790257629041645-13911-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790257629041645-13911-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790257629041645-13911-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.149.193:41099
--local_ip_for_outbound_sockets=127.13.149.193
--tserver_master_addrs=127.13.149.254:45043
--webserver_port=42325
--webserver_interface=127.13.149.193
--keytab_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.193.keytab
--principal=kudu/127.13.149.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.149.212:45653
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.13.149.193
--keytab_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790257629041645-13911-0/krb5kdc/oryx_127.13.149.193.keytab with env {KRB5CCNAME=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790257629041645-13911-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790257629041645-13911-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790257629041645-13911-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260924 13:48:14.434633 22745 log.cc:824] T 00000000000000000000000000000000 P 94bf763b8fce4015898b8ee17c4faf3e: Log is configured to *not* fsync() on all Append() calls
I20260924 13:48:14.436426 22745 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 94bf763b8fce4015898b8ee17c4faf3e: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260924 13:48:14.436694 22745 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 94bf763b8fce4015898b8ee17c4faf3e: Bootstrap complete.
I20260924 13:48:14.438820 22745 raft_consensus.cc:348] T 00000000000000000000000000000000 P 94bf763b8fce4015898b8ee17c4faf3e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "94bf763b8fce4015898b8ee17c4faf3e" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 45043 } }
I20260924 13:48:14.439062 22745 raft_consensus.cc:729] T 00000000000000000000000000000000 P 94bf763b8fce4015898b8ee17c4faf3e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 94bf763b8fce4015898b8ee17c4faf3e, State: Initialized, Role: FOLLOWER
I20260924 13:48:14.439205 22745 consensus_queue.cc:260] T 00000000000000000000000000000000 P 94bf763b8fce4015898b8ee17c4faf3e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "94bf763b8fce4015898b8ee17c4faf3e" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 45043 } }
I20260924 13:48:14.439271 22745 raft_consensus.cc:388] T 00000000000000000000000000000000 P 94bf763b8fce4015898b8ee17c4faf3e [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 13:48:14.439312 22745 raft_consensus.cc:482] T 00000000000000000000000000000000 P 94bf763b8fce4015898b8ee17c4faf3e [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 13:48:14.439357 22745 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 94bf763b8fce4015898b8ee17c4faf3e [term 1 FOLLOWER]: Advancing to term 2
I20260924 13:48:14.440336 22745 raft_consensus.cc:504] T 00000000000000000000000000000000 P 94bf763b8fce4015898b8ee17c4faf3e [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "94bf763b8fce4015898b8ee17c4faf3e" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 45043 } }
I20260924 13:48:14.440423 22745 leader_election.cc:302] T 00000000000000000000000000000000 P 94bf763b8fce4015898b8ee17c4faf3e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 94bf763b8fce4015898b8ee17c4faf3e; no voters:
I20260924 13:48:14.440608 22745 leader_election.cc:288] T 00000000000000000000000000000000 P 94bf763b8fce4015898b8ee17c4faf3e [CANDIDATE]: Term 2 election: Requested vote from peers
I20260924 13:48:14.440657 22749 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 94bf763b8fce4015898b8ee17c4faf3e [term 2 FOLLOWER]: Leader election won for term 2
I20260924 13:48:14.440828 22749 raft_consensus.cc:686] T 00000000000000000000000000000000 P 94bf763b8fce4015898b8ee17c4faf3e [term 2 LEADER]: Becoming Leader. State: Replica: 94bf763b8fce4015898b8ee17c4faf3e, State: Running, Role: LEADER
I20260924 13:48:14.440872 22745 sys_catalog.cc:564] T 00000000000000000000000000000000 P 94bf763b8fce4015898b8ee17c4faf3e [sys.catalog]: configured and running, proceeding with master startup.
I20260924 13:48:14.441015 22749 consensus_queue.cc:237] T 00000000000000000000000000000000 P 94bf763b8fce4015898b8ee17c4faf3e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "94bf763b8fce4015898b8ee17c4faf3e" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 45043 } }
I20260924 13:48:14.441356 22751 sys_catalog.cc:455] T 00000000000000000000000000000000 P 94bf763b8fce4015898b8ee17c4faf3e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 94bf763b8fce4015898b8ee17c4faf3e. Latest consensus state: current_term: 2 leader_uuid: "94bf763b8fce4015898b8ee17c4faf3e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "94bf763b8fce4015898b8ee17c4faf3e" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 45043 } } }
I20260924 13:48:14.441363 22750 sys_catalog.cc:455] T 00000000000000000000000000000000 P 94bf763b8fce4015898b8ee17c4faf3e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "94bf763b8fce4015898b8ee17c4faf3e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "94bf763b8fce4015898b8ee17c4faf3e" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 45043 } } }
I20260924 13:48:14.441525 22750 sys_catalog.cc:458] T 00000000000000000000000000000000 P 94bf763b8fce4015898b8ee17c4faf3e [sys.catalog]: This master's current role is: LEADER
I20260924 13:48:14.441448 22751 sys_catalog.cc:458] T 00000000000000000000000000000000 P 94bf763b8fce4015898b8ee17c4faf3e [sys.catalog]: This master's current role is: LEADER
I20260924 13:48:14.441970 22758 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 13:48:14.442526 22758 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 13:48:14.442667 22758 catalog_manager.cc:1259] Loaded cluster ID: fb9fd0abb2144a448e7a4acccf23e91f
I20260924 13:48:14.442699 22758 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 13:48:14.443892 22758 catalog_manager.cc:1504] Loading token signing keys...
I20260924 13:48:14.444056 22758 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 94bf763b8fce4015898b8ee17c4faf3e: Loaded TSK: 0
I20260924 13:48:14.444259 22758 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260924 13:48:14.514642 22747 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 13:48:14.514854 22747 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 13:48:14.514879 22747 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 13:48:14.516080 22747 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 13:48:14.516121 22747 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 13:48:14.516147 22747 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.149.193
I20260924 13:48:14.517656 22747 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.149.212:45653
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790257629041645-13911-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790257629041645-13911-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790257629041645-13911-0/krb5kdc/oryx_127.13.149.193.keytab
--principal=oryx/127.13.149.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.149.193:41099
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790257629041645-13911-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.149.193
--webserver_port=42325
--tserver_master_addrs=127.13.149.254:45043
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22747
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.149.193
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790257629041645-13911-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 12:29:37 UTC on ccycloud.quasar-iqzbxn.root.comops.site
build id 4670106
I20260924 13:48:14.517884 22747 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 13:48:14.518106 22747 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 13:48:14.522970 22771 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 13:48:14.522982 22774 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 13:48:14.523121 22747 server_base.cc:1034] running on GCE node
W20260924 13:48:14.522985 22772 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 13:48:14.523439 22747 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 13:48:14.523672 22747 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 13:48:14.524820 22747 hybrid_clock.cc:648] HybridClock initialized: now 1790257694524781 us; error 55 us; skew 500 ppm
Sep 24 13:48:14 dist-test-slave-sjl2 krb5kdc[22164](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257694, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.149.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 13:48:14.527551 22747 init.cc:377] Logged in from keytab as oryx/127.13.149.193@KRBTEST.COM (short username oryx)
I20260924 13:48:14.528112 22747 webserver.cc:460] Webserver started at http://127.13.149.193:42325/ using document root <none> and password file <none>
I20260924 13:48:14.528368 22747 fs_manager.cc:359] Metadata directory not provided
I20260924 13:48:14.528446 22747 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 13:48:14.530200 22747 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260924 13:48:14.530926 22781 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 13:48:14.531114 22747 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260924 13:48:14.531211 22747 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790257629041645-13911-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790257629041645-13911-0/minicluster-data/ts-0/wal
uuid: "2842fbd855834006a6b4ca8c5a11db24"
format_stamp: "Formatted at 2026-09-24 13:48:13 on dist-test-slave-sjl2"
server_key: "8a6783ebdb24ebf935507b8db2c3b833"
server_key_iv: "fe0fc849df5ce01c094f453374e11398"
server_key_version: "encryptionkey@0"
I20260924 13:48:14.531667 22747 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790257629041645-13911-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790257629041645-13911-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790257629041645-13911-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 13:48:14.558485 22747 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 13:48:14.560890 22747 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 13:48:14.561089 22747 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 13:48:14.561339 22747 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 13:48:14.561671 22747 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 13:48:14.561717 22747 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 13:48:14.561743 22747 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 13:48:14.561764 22747 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 13:48:14.568712 22747 rpc_server.cc:307] RPC server started. Bound to: 127.13.149.193:41099
I20260924 13:48:14.568754 22894 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.149.193:41099 every 8 connection(s)
I20260924 13:48:14.569319 22747 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790257629041645-13911-0/minicluster-data/ts-0/data/info.pb
Sep 24 13:48:14 dist-test-slave-sjl2 krb5kdc[22164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.149.193@KRBTEST.COM for oryx/127.13.149.254@KRBTEST.COM, Server not found in Kerberos database
Sep 24 13:48:14 dist-test-slave-sjl2 krb5kdc[22164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.149.193@KRBTEST.COM for oryx/127.13.149.254@KRBTEST.COM, Server not found in Kerberos database
I20260924 13:48:14.577420 13911 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJN9WRF/build/release/bin/kudu as pid 22747
I20260924 13:48:14.577700 13911 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJN9WRF/build/release/bin/kudu
/tmp/dist-test-taskJN9WRF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790257629041645-13911-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790257629041645-13911-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790257629041645-13911-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790257629041645-13911-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.149.194:42801
--local_ip_for_outbound_sockets=127.13.149.194
--tserver_master_addrs=127.13.149.254:45043
--webserver_port=42049
--webserver_interface=127.13.149.194
--keytab_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.194.keytab
--principal=kudu/127.13.149.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.149.212:45653
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.13.149.194
--keytab_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790257629041645-13911-0/krb5kdc/oryx_127.13.149.194.keytab with env {KRB5CCNAME=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790257629041645-13911-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790257629041645-13911-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790257629041645-13911-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260924 13:48:14.577865 22897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:48:14.570233 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:45043 (local address 127.13.149.193:58049)
0924 13:48:14.570423 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0924 13:48:14.570474 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0924 13:48:14.570974 (+ 500us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:48:14.571381 (+ 407us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:48:14.571385 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:48:14.571965 (+ 580us) client_negotiation.cc:189] Negotiated authn=SASL
0924 13:48:14.572155 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:48:14.572160 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:48:14.573273 (+ 1113us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:48:14.573274 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:48:14.574044 (+ 770us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:48:14.574047 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:48:14.574596 (+ 549us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:48:14.574597 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:48:14.574720 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:48:14.575255 (+ 535us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 13:48:14.575263 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0924 13:48:14.577672 (+ 2409us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.149.254:45043: Server oryx/127.13.149.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":82,"threads_started":1}
W20260924 13:48:14.578105 22895 heartbeater.cc:644] Failed to heartbeat to 127.13.149.254:45043 (0 consecutive failures): Not authorized: Failed to ping master at 127.13.149.254:45043: Client connection negotiation failed: client connection to 127.13.149.254:45043: Server oryx/127.13.149.254@KRBTEST.COM not found in Kerberos database
W20260924 13:48:14.578251 22898 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 13:48:14.570285 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.149.193:58049 (local address 127.13.149.254:45043)
0924 13:48:14.570518 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0924 13:48:14.570525 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0924 13:48:14.570559 (+ 34us) server_negotiation.cc:407] Connection header received
0924 13:48:14.571174 (+ 615us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:48:14.571179 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:48:14.571220 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:48:14.571331 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:48:14.572271 (+ 940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:48:14.573186 (+ 915us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:48:14.574131 (+ 945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:48:14.574521 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:48:14.574611 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:48:14.577837 (+ 3226us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.149.193:58049: BlockingRecv error: recv got EOF from 127.13.149.193:58049 (error 108)
Metrics: {"server-negotiator.queue_time_us":190,"thread_start_us":81,"threads_started":1}
Sep 24 13:48:14 dist-test-slave-sjl2 krb5kdc[22164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.149.193@KRBTEST.COM for oryx/127.13.149.254@KRBTEST.COM, Server not found in Kerberos database
Sep 24 13:48:14 dist-test-slave-sjl2 krb5kdc[22164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.149.193@KRBTEST.COM for oryx/127.13.149.254@KRBTEST.COM, Server not found in Kerberos database
I20260924 13:48:14.583501 22897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:48:14.578362 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:45043 (local address 127.13.149.193:34335)
0924 13:48:14.578442 (+ 80us) negotiation.cc:107] Waiting for socket to connect
0924 13:48:14.578454 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0924 13:48:14.578487 (+ 33us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:48:14.578786 (+ 299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:48:14.578787 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:48:14.578927 (+ 140us) client_negotiation.cc:189] Negotiated authn=SASL
0924 13:48:14.578997 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:48:14.578999 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:48:14.579870 (+ 871us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:48:14.579871 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:48:14.580581 (+ 710us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:48:14.580584 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:48:14.580936 (+ 352us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:48:14.580937 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:48:14.581003 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:48:14.581435 (+ 432us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 13:48:14.581440 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 13:48:14.583393 (+ 1953us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.149.254:45043: Server oryx/127.13.149.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":65}
W20260924 13:48:14.583552 22898 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 13:48:14.578608 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.149.193:34335 (local address 127.13.149.254:45043)
0924 13:48:14.578660 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0924 13:48:14.578662 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:48:14.578669 (+ 7us) server_negotiation.cc:407] Connection header received
0924 13:48:14.578691 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:48:14.578692 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:48:14.578704 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:48:14.578745 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:48:14.579075 (+ 330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:48:14.579774 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:48:14.580665 (+ 891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:48:14.580848 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:48:14.580898 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:48:14.583499 (+ 2601us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.149.193:34335: BlockingRecv error: recv got EOF from 127.13.149.193:34335 (error 108)
Metrics: {"server-negotiator.queue_time_us":33}
Sep 24 13:48:14 dist-test-slave-sjl2 krb5kdc[22164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.149.193@KRBTEST.COM for oryx/127.13.149.254@KRBTEST.COM, Server not found in Kerberos database
Sep 24 13:48:14 dist-test-slave-sjl2 krb5kdc[22164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.149.193@KRBTEST.COM for oryx/127.13.149.254@KRBTEST.COM, Server not found in Kerberos database
I20260924 13:48:14.588085 22897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:48:14.583815 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:45043 (local address 127.13.149.193:56895)
0924 13:48:14.583861 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0924 13:48:14.583872 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0924 13:48:14.583913 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:48:14.584050 (+ 137us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:48:14.584051 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:48:14.584167 (+ 116us) client_negotiation.cc:189] Negotiated authn=SASL
0924 13:48:14.584198 (+ 31us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:48:14.584199 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:48:14.584805 (+ 606us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:48:14.584806 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:48:14.585351 (+ 545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:48:14.585353 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:48:14.585669 (+ 316us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:48:14.585670 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:48:14.585718 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:48:14.586211 (+ 493us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 13:48:14.586216 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 13:48:14.588029 (+ 1813us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.149.254:45043: Server oryx/127.13.149.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":28}
W20260924 13:48:14.588202 22898 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 13:48:14.583889 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.149.193:56895 (local address 127.13.149.254:45043)
0924 13:48:14.583954 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0924 13:48:14.583955 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0924 13:48:14.583962 (+ 7us) server_negotiation.cc:407] Connection header received
0924 13:48:14.583985 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:48:14.583986 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:48:14.583998 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:48:14.584041 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:48:14.584259 (+ 218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:48:14.584735 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:48:14.585429 (+ 694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:48:14.585614 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:48:14.585668 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:48:14.588140 (+ 2472us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.149.193:56895: BlockingRecv error: recv got EOF from 127.13.149.193:56895 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
W20260924 13:48:14.588241 22895 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260924 13:48:14.661280 22899 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 13:48:14.661509 22899 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 13:48:14.661532 22899 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 13:48:14.662715 22899 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 13:48:14.662752 22899 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 13:48:14.662779 22899 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.149.194
I20260924 13:48:14.664258 22899 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.149.212:45653
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790257629041645-13911-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790257629041645-13911-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790257629041645-13911-0/krb5kdc/oryx_127.13.149.194.keytab
--principal=oryx/127.13.149.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.149.194:42801
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790257629041645-13911-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.149.194
--webserver_port=42049
--tserver_master_addrs=127.13.149.254:45043
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22899
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.149.194
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790257629041645-13911-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 12:29:37 UTC on ccycloud.quasar-iqzbxn.root.comops.site
build id 4670106
I20260924 13:48:14.664465 22899 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 13:48:14.664664 22899 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 13:48:14.669569 22905 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 13:48:14.669605 22904 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 13:48:14.669561 22907 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 13:48:14.669776 22899 server_base.cc:1034] running on GCE node
I20260924 13:48:14.670043 22899 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 13:48:14.670310 22899 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 13:48:14.671507 22899 hybrid_clock.cc:648] HybridClock initialized: now 1790257694671488 us; error 35 us; skew 500 ppm
Sep 24 13:48:14 dist-test-slave-sjl2 krb5kdc[22164](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257694, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.149.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 13:48:14.673969 22899 init.cc:377] Logged in from keytab as oryx/127.13.149.194@KRBTEST.COM (short username oryx)
I20260924 13:48:14.674401 22899 webserver.cc:460] Webserver started at http://127.13.149.194:42049/ using document root <none> and password file <none>
I20260924 13:48:14.674603 22899 fs_manager.cc:359] Metadata directory not provided
I20260924 13:48:14.674662 22899 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 13:48:14.676066 22899 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260924 13:48:14.676622 22914 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 13:48:14.676782 22899 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260924 13:48:14.676864 22899 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790257629041645-13911-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790257629041645-13911-0/minicluster-data/ts-1/wal
uuid: "d0687009a62c4ec7a045f6856ef0be30"
format_stamp: "Formatted at 2026-09-24 13:48:14 on dist-test-slave-sjl2"
server_key: "913f4420175e9fe624c4863c06b58e59"
server_key_iv: "3b2a90f909090b9443a1c8a9213a7f0a"
server_key_version: "encryptionkey@0"
I20260924 13:48:14.677261 22899 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790257629041645-13911-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790257629041645-13911-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790257629041645-13911-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 13:48:14.688866 22899 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 13:48:14.690979 22899 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 13:48:14.691099 22899 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 13:48:14.691283 22899 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 13:48:14.691562 22899 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 13:48:14.691594 22899 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 13:48:14.691617 22899 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 13:48:14.691640 22899 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 13:48:14.697090 22899 rpc_server.cc:307] RPC server started. Bound to: 127.13.149.194:42801
I20260924 13:48:14.697137 23027 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.149.194:42801 every 8 connection(s)
I20260924 13:48:14.697443 22899 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790257629041645-13911-0/minicluster-data/ts-1/data/info.pb
Sep 24 13:48:14 dist-test-slave-sjl2 krb5kdc[22164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.149.194@KRBTEST.COM for oryx/127.13.149.254@KRBTEST.COM, Server not found in Kerberos database
I20260924 13:48:14.703125 13911 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJN9WRF/build/release/bin/kudu as pid 22899
I20260924 13:48:14.703390 13911 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJN9WRF/build/release/bin/kudu
/tmp/dist-test-taskJN9WRF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790257629041645-13911-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790257629041645-13911-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790257629041645-13911-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790257629041645-13911-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.149.195:42947
--local_ip_for_outbound_sockets=127.13.149.195
--tserver_master_addrs=127.13.149.254:45043
--webserver_port=39967
--webserver_interface=127.13.149.195
--keytab_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.195.keytab
--principal=kudu/127.13.149.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.149.212:45653
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.13.149.195
--keytab_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790257629041645-13911-0/krb5kdc/oryx_127.13.149.195.keytab with env {KRB5CCNAME=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790257629041645-13911-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790257629041645-13911-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790257629041645-13911-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 24 13:48:14 dist-test-slave-sjl2 krb5kdc[22164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.149.194@KRBTEST.COM for oryx/127.13.149.254@KRBTEST.COM, Server not found in Kerberos database
W20260924 13:48:14.704562 22898 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 13:48:14.698120 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.149.194:39163 (local address 127.13.149.254:45043)
0924 13:48:14.698246 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0924 13:48:14.698248 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:48:14.698435 (+ 187us) server_negotiation.cc:407] Connection header received
0924 13:48:14.698926 (+ 491us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:48:14.698927 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:48:14.698943 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:48:14.698994 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:48:14.699926 (+ 932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:48:14.700412 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:48:14.701405 (+ 993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:48:14.701604 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:48:14.701658 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:48:14.704502 (+ 2844us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.149.194:39163: BlockingRecv error: recv got EOF from 127.13.149.194:39163 (error 108)
Metrics: {"server-negotiator.queue_time_us":91}
I20260924 13:48:14.704630 23030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:48:14.698103 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:45043 (local address 127.13.149.194:39163)
0924 13:48:14.698318 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0924 13:48:14.698356 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0924 13:48:14.698836 (+ 480us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:48:14.699060 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:48:14.699063 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:48:14.699645 (+ 582us) client_negotiation.cc:189] Negotiated authn=SASL
0924 13:48:14.699836 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:48:14.699841 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:48:14.700495 (+ 654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:48:14.700496 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:48:14.701327 (+ 831us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:48:14.701329 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:48:14.701671 (+ 342us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:48:14.701672 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:48:14.701785 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:48:14.702246 (+ 461us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 13:48:14.702255 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0924 13:48:14.704391 (+ 2136us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.149.254:45043: Server oryx/127.13.149.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":74,"threads_started":1}
W20260924 13:48:14.704874 23028 heartbeater.cc:644] Failed to heartbeat to 127.13.149.254:45043 (0 consecutive failures): Not authorized: Failed to ping master at 127.13.149.254:45043: Client connection negotiation failed: client connection to 127.13.149.254:45043: Server oryx/127.13.149.254@KRBTEST.COM not found in Kerberos database
Sep 24 13:48:14 dist-test-slave-sjl2 krb5kdc[22164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.149.194@KRBTEST.COM for oryx/127.13.149.254@KRBTEST.COM, Server not found in Kerberos database
Sep 24 13:48:14 dist-test-slave-sjl2 krb5kdc[22164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.149.194@KRBTEST.COM for oryx/127.13.149.254@KRBTEST.COM, Server not found in Kerberos database
I20260924 13:48:14.709137 23030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:48:14.705130 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:45043 (local address 127.13.149.194:37147)
0924 13:48:14.705177 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0924 13:48:14.705187 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 13:48:14.705230 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:48:14.705346 (+ 116us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:48:14.705347 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:48:14.705472 (+ 125us) client_negotiation.cc:189] Negotiated authn=SASL
0924 13:48:14.705519 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:48:14.705521 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:48:14.706130 (+ 609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:48:14.706131 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:48:14.706696 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:48:14.706698 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:48:14.707012 (+ 314us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:48:14.707013 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:48:14.707082 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:48:14.707469 (+ 387us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 13:48:14.707475 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0924 13:48:14.709076 (+ 1601us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.149.254:45043: Server oryx/127.13.149.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":29}
W20260924 13:48:14.709281 22898 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 13:48:14.705185 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.149.194:37147 (local address 127.13.149.254:45043)
0924 13:48:14.705230 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0924 13:48:14.705232 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:48:14.705241 (+ 9us) server_negotiation.cc:407] Connection header received
0924 13:48:14.705289 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:48:14.705290 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:48:14.705300 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:48:14.705344 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:48:14.705586 (+ 242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:48:14.706062 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:48:14.706769 (+ 707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:48:14.706956 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:48:14.707008 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:48:14.709220 (+ 2212us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.149.194:37147: BlockingRecv error: recv got EOF from 127.13.149.194:37147 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
Sep 24 13:48:14 dist-test-slave-sjl2 krb5kdc[22164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.149.194@KRBTEST.COM for oryx/127.13.149.254@KRBTEST.COM, Server not found in Kerberos database
Sep 24 13:48:14 dist-test-slave-sjl2 krb5kdc[22164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.149.194@KRBTEST.COM for oryx/127.13.149.254@KRBTEST.COM, Server not found in Kerberos database
I20260924 13:48:14.714110 23030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:48:14.709408 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:45043 (local address 127.13.149.194:48225)
0924 13:48:14.709450 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0924 13:48:14.709460 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 13:48:14.709497 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:48:14.709718 (+ 221us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:48:14.709719 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:48:14.709832 (+ 113us) client_negotiation.cc:189] Negotiated authn=SASL
0924 13:48:14.709871 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:48:14.709873 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:48:14.710786 (+ 913us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:48:14.710787 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:48:14.711277 (+ 490us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:48:14.711279 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:48:14.711709 (+ 430us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:48:14.711710 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:48:14.711746 (+ 36us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:48:14.712188 (+ 442us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 13:48:14.712192 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0924 13:48:14.714034 (+ 1842us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.149.254:45043: Server oryx/127.13.149.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":27}
W20260924 13:48:14.714188 22898 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 13:48:14.709467 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.149.194:48225 (local address 127.13.149.254:45043)
0924 13:48:14.709567 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0924 13:48:14.709569 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:48:14.709579 (+ 10us) server_negotiation.cc:407] Connection header received
0924 13:48:14.709605 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:48:14.709606 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:48:14.709617 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:48:14.709688 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:48:14.709960 (+ 272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:48:14.710703 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:48:14.711358 (+ 655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:48:14.711642 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:48:14.711712 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:48:14.714123 (+ 2411us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.149.194:48225: BlockingRecv error: recv got EOF from 127.13.149.194:48225 (error 108)
Metrics: {"server-negotiator.queue_time_us":78}
W20260924 13:48:14.714269 23028 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260924 13:48:14.784855 23031 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 13:48:14.785099 23031 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 13:48:14.785125 23031 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 13:48:14.786373 23031 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 13:48:14.786430 23031 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 13:48:14.786459 23031 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.149.195
I20260924 13:48:14.788008 23031 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.149.212:45653
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790257629041645-13911-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790257629041645-13911-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790257629041645-13911-0/krb5kdc/oryx_127.13.149.195.keytab
--principal=oryx/127.13.149.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.149.195:42947
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790257629041645-13911-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.149.195
--webserver_port=39967
--tserver_master_addrs=127.13.149.254:45043
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23031
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.149.195
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790257629041645-13911-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 12:29:37 UTC on ccycloud.quasar-iqzbxn.root.comops.site
build id 4670106
I20260924 13:48:14.788265 23031 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 13:48:14.788491 23031 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 13:48:14.793660 23037 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 13:48:14.793666 23036 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 13:48:14.793684 23031 server_base.cc:1034] running on GCE node
W20260924 13:48:14.793684 23039 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 13:48:14.794098 23031 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 13:48:14.794322 23031 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 13:48:14.795521 23031 hybrid_clock.cc:648] HybridClock initialized: now 1790257694795509 us; error 29 us; skew 500 ppm
Sep 24 13:48:14 dist-test-slave-sjl2 krb5kdc[22164](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257694, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.149.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 13:48:14.798141 23031 init.cc:377] Logged in from keytab as oryx/127.13.149.195@KRBTEST.COM (short username oryx)
I20260924 13:48:14.798624 23031 webserver.cc:460] Webserver started at http://127.13.149.195:39967/ using document root <none> and password file <none>
I20260924 13:48:14.798846 23031 fs_manager.cc:359] Metadata directory not provided
I20260924 13:48:14.798905 23031 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 13:48:14.800302 23031 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260924 13:48:14.800964 23046 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 13:48:14.801129 23031 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260924 13:48:14.801213 23031 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790257629041645-13911-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790257629041645-13911-0/minicluster-data/ts-2/wal
uuid: "d3f5f2aaae334a7ebf34d1872226b5a5"
format_stamp: "Formatted at 2026-09-24 13:48:14 on dist-test-slave-sjl2"
server_key: "9b07c7f9152f1097a2c49c1a4e67447d"
server_key_iv: "7ae450f58e8979bd161a4e2ff19140da"
server_key_version: "encryptionkey@0"
I20260924 13:48:14.801581 23031 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790257629041645-13911-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790257629041645-13911-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790257629041645-13911-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 13:48:14.823522 23031 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 13:48:14.825611 23031 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 13:48:14.825760 23031 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 13:48:14.825976 23031 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 13:48:14.826265 23031 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 13:48:14.826299 23031 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 13:48:14.826327 23031 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 13:48:14.826345 23031 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 13:48:14.831729 23031 rpc_server.cc:307] RPC server started. Bound to: 127.13.149.195:42947
I20260924 13:48:14.831782 23159 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.149.195:42947 every 8 connection(s)
I20260924 13:48:14.832111 23031 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790257629041645-13911-0/minicluster-data/ts-2/data/info.pb
Sep 24 13:48:14 dist-test-slave-sjl2 krb5kdc[22164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.149.195@KRBTEST.COM for oryx/127.13.149.254@KRBTEST.COM, Server not found in Kerberos database
Sep 24 13:48:14 dist-test-slave-sjl2 krb5kdc[22164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.149.195@KRBTEST.COM for oryx/127.13.149.254@KRBTEST.COM, Server not found in Kerberos database
I20260924 13:48:14.838610 13911 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJN9WRF/build/release/bin/kudu as pid 23031
W20260924 13:48:14.839370 22898 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 13:48:14.832818 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.149.195:57877 (local address 127.13.149.254:45043)
0924 13:48:14.832864 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0924 13:48:14.832866 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:48:14.833106 (+ 240us) server_negotiation.cc:407] Connection header received
0924 13:48:14.833616 (+ 510us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:48:14.833618 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:48:14.833630 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:48:14.833681 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:48:14.834619 (+ 938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:48:14.835127 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:48:14.836158 (+ 1031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:48:14.836344 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:48:14.836399 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:48:14.839315 (+ 2916us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.149.195:57877: BlockingRecv error: recv got EOF from 127.13.149.195:57877 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
I20260924 13:48:14.839475 23162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:48:14.832813 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:45043 (local address 127.13.149.195:57877)
0924 13:48:14.832976 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0924 13:48:14.833017 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0924 13:48:14.833516 (+ 499us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:48:14.833751 (+ 235us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:48:14.833755 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:48:14.834326 (+ 571us) client_negotiation.cc:189] Negotiated authn=SASL
0924 13:48:14.834521 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:48:14.834526 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:48:14.835236 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:48:14.835238 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:48:14.836080 (+ 842us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:48:14.836083 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:48:14.836412 (+ 329us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:48:14.836413 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:48:14.836539 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:48:14.837075 (+ 536us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 13:48:14.837084 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0924 13:48:14.839234 (+ 2150us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.149.254:45043: Server oryx/127.13.149.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":79,"threads_started":1}
W20260924 13:48:14.839720 23160 heartbeater.cc:644] Failed to heartbeat to 127.13.149.254:45043 (0 consecutive failures): Not authorized: Failed to ping master at 127.13.149.254:45043: Client connection negotiation failed: client connection to 127.13.149.254:45043: Server oryx/127.13.149.254@KRBTEST.COM not found in Kerberos database
Sep 24 13:48:14 dist-test-slave-sjl2 krb5kdc[22164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.149.195@KRBTEST.COM for oryx/127.13.149.254@KRBTEST.COM, Server not found in Kerberos database
Sep 24 13:48:14 dist-test-slave-sjl2 krb5kdc[22164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.149.195@KRBTEST.COM for oryx/127.13.149.254@KRBTEST.COM, Server not found in Kerberos database
I20260924 13:48:14.844192 23162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:48:14.839907 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:45043 (local address 127.13.149.195:36651)
0924 13:48:14.839948 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0924 13:48:14.839956 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0924 13:48:14.839990 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:48:14.840091 (+ 101us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:48:14.840092 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:48:14.840248 (+ 156us) client_negotiation.cc:189] Negotiated authn=SASL
0924 13:48:14.840296 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:48:14.840298 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:48:14.840965 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:48:14.840966 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:48:14.841767 (+ 801us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:48:14.841770 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:48:14.842175 (+ 405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:48:14.842176 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:48:14.842250 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:48:14.842541 (+ 291us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 13:48:14.842545 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0924 13:48:14.844123 (+ 1578us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.149.254:45043: Server oryx/127.13.149.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":28}
W20260924 13:48:14.844257 22898 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 13:48:14.839950 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.149.195:36651 (local address 127.13.149.254:45043)
0924 13:48:14.839988 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0924 13:48:14.839990 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:48:14.839997 (+ 7us) server_negotiation.cc:407] Connection header received
0924 13:48:14.840041 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:48:14.840042 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:48:14.840048 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:48:14.840084 (+ 36us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:48:14.840357 (+ 273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:48:14.840869 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:48:14.841846 (+ 977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:48:14.842109 (+ 263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:48:14.842164 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:48:14.844207 (+ 2043us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.149.195:36651: BlockingRecv error: recv got EOF from 127.13.149.195:36651 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
I20260924 13:48:14.845923 23164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:48:14.839096 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57922 (local address 127.13.149.254:45043)
0924 13:48:14.839196 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0924 13:48:14.839197 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0924 13:48:14.839208 (+ 11us) server_negotiation.cc:407] Connection header received
0924 13:48:14.839281 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:48:14.839282 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:48:14.839293 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:48:14.839347 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:48:14.839965 (+ 618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:48:14.840603 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:48:14.841472 (+ 869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:48:14.841781 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:48:14.841840 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:48:14.843069 (+ 1229us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:48:14.843073 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:48:14.843073 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 13:48:14.843094 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:48:14.844680 (+ 1586us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:48:14.845014 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:48:14.845017 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:48:14.845018 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:48:14.845105 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:48:14.845276 (+ 171us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:48:14.845277 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:48:14.845277 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:48:14.845496 (+ 219us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:48:14.845619 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0924 13:48:14.845729 (+ 110us) server_negotiation.cc:299] Negotiation successful
0924 13:48:14.845834 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81,"thread_start_us":50,"threads_started":1}
Sep 24 13:48:14 dist-test-slave-sjl2 krb5kdc[22164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.149.195@KRBTEST.COM for oryx/127.13.149.254@KRBTEST.COM, Server not found in Kerberos database
Sep 24 13:48:14 dist-test-slave-sjl2 krb5kdc[22164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.149.195@KRBTEST.COM for oryx/127.13.149.254@KRBTEST.COM, Server not found in Kerberos database
I20260924 13:48:14.848397 23162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:48:14.844418 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:45043 (local address 127.13.149.195:34733)
0924 13:48:14.844542 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0924 13:48:14.844551 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0924 13:48:14.844593 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:48:14.844700 (+ 107us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:48:14.844701 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:48:14.844806 (+ 105us) client_negotiation.cc:189] Negotiated authn=SASL
0924 13:48:14.844833 (+ 27us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:48:14.844834 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:48:14.845431 (+ 597us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:48:14.845432 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:48:14.845924 (+ 492us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:48:14.845926 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:48:14.846554 (+ 628us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:48:14.846555 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:48:14.846609 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:48:14.846943 (+ 334us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 13:48:14.846947 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0924 13:48:14.848344 (+ 1397us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.149.254:45043: Server oryx/127.13.149.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":82,"threads_started":1}
W20260924 13:48:14.848531 23160 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260924 13:48:14.848548 22898 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 13:48:14.844469 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.149.195:34733 (local address 127.13.149.254:45043)
0924 13:48:14.844503 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0924 13:48:14.844504 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0924 13:48:14.844584 (+ 80us) server_negotiation.cc:407] Connection header received
0924 13:48:14.844635 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:48:14.844636 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:48:14.844645 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:48:14.844690 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:48:14.844891 (+ 201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:48:14.845368 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:48:14.846197 (+ 829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:48:14.846473 (+ 276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:48:14.846533 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:48:14.848493 (+ 1960us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.149.195:34733: BlockingRecv error: recv got EOF from 127.13.149.195:34733 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
Sep 24 13:48:15 dist-test-slave-sjl2 krb5kdc[22164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.149.193@KRBTEST.COM for oryx/127.13.149.254@KRBTEST.COM, Server not found in Kerberos database
Sep 24 13:48:15 dist-test-slave-sjl2 krb5kdc[22164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.149.193@KRBTEST.COM for oryx/127.13.149.254@KRBTEST.COM, Server not found in Kerberos database
I20260924 13:48:15.593505 23166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:48:15.588690 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:45043 (local address 127.13.149.193:48797)
0924 13:48:15.588828 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0924 13:48:15.588840 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0924 13:48:15.588904 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:48:15.589102 (+ 198us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:48:15.589103 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:48:15.589305 (+ 202us) client_negotiation.cc:189] Negotiated authn=SASL
0924 13:48:15.589370 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:48:15.589372 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:48:15.590080 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:48:15.590081 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:48:15.590660 (+ 579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:48:15.590662 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:48:15.591042 (+ 380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:48:15.591043 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:48:15.591098 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:48:15.591582 (+ 484us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 13:48:15.591588 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0924 13:48:15.593432 (+ 1844us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.149.254:45043: Server oryx/127.13.149.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":65,"threads_started":1}
W20260924 13:48:15.593657 23167 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 13:48:15.588858 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.149.193:48797 (local address 127.13.149.254:45043)
0924 13:48:15.588962 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0924 13:48:15.588964 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:48:15.588990 (+ 26us) server_negotiation.cc:407] Connection header received
0924 13:48:15.589012 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:48:15.589013 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:48:15.589025 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:48:15.589088 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:48:15.589444 (+ 356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:48:15.589973 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:48:15.590763 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:48:15.590977 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:48:15.591036 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:48:15.593576 (+ 2540us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.149.193:48797: BlockingRecv error: recv got EOF from 127.13.149.193:48797 (error 108)
Metrics: {"server-negotiator.queue_time_us":82,"thread_start_us":41,"threads_started":1}
Sep 24 13:48:15 dist-test-slave-sjl2 krb5kdc[22164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.149.194@KRBTEST.COM for oryx/127.13.149.254@KRBTEST.COM, Server not found in Kerberos database
Sep 24 13:48:15 dist-test-slave-sjl2 krb5kdc[22164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.149.194@KRBTEST.COM for oryx/127.13.149.254@KRBTEST.COM, Server not found in Kerberos database
I20260924 13:48:15.720503 23168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:48:15.714825 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:45043 (local address 127.13.149.194:47211)
0924 13:48:15.715028 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0924 13:48:15.715043 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0924 13:48:15.715118 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:48:15.715328 (+ 210us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:48:15.715329 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:48:15.715492 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0924 13:48:15.715564 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:48:15.715566 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:48:15.716268 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:48:15.716269 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:48:15.717095 (+ 826us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:48:15.717100 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:48:15.717505 (+ 405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:48:15.717507 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:48:15.717558 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:48:15.718327 (+ 769us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 13:48:15.718333 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0924 13:48:15.720408 (+ 2075us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.149.254:45043: Server oryx/127.13.149.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":78,"threads_started":1}
W20260924 13:48:15.720608 23167 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 13:48:15.714959 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.149.194:47211 (local address 127.13.149.254:45043)
0924 13:48:15.715030 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0924 13:48:15.715032 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:48:15.715109 (+ 77us) server_negotiation.cc:407] Connection header received
0924 13:48:15.715179 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:48:15.715180 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:48:15.715192 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:48:15.715233 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:48:15.715663 (+ 430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:48:15.716169 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:48:15.717197 (+ 1028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:48:15.717410 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:48:15.717474 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:48:15.720543 (+ 3069us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.149.194:47211: BlockingRecv error: recv got EOF from 127.13.149.194:47211 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
Sep 24 13:48:15 dist-test-slave-sjl2 krb5kdc[22164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.149.195@KRBTEST.COM for oryx/127.13.149.254@KRBTEST.COM, Server not found in Kerberos database
Sep 24 13:48:15 dist-test-slave-sjl2 krb5kdc[22164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.149.195@KRBTEST.COM for oryx/127.13.149.254@KRBTEST.COM, Server not found in Kerberos database
I20260924 13:48:15.853470 23169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:48:15.848982 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:45043 (local address 127.13.149.195:49171)
0924 13:48:15.849138 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0924 13:48:15.849149 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0924 13:48:15.849218 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:48:15.849367 (+ 149us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:48:15.849368 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:48:15.849528 (+ 160us) client_negotiation.cc:189] Negotiated authn=SASL
0924 13:48:15.849577 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:48:15.849579 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:48:15.850220 (+ 641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:48:15.850221 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:48:15.850799 (+ 578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:48:15.850801 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:48:15.851112 (+ 311us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:48:15.851113 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:48:15.851158 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:48:15.851651 (+ 493us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 13:48:15.851657 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0924 13:48:15.853380 (+ 1723us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.149.254:45043: Server oryx/127.13.149.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":80,"threads_started":1}
W20260924 13:48:15.853549 23167 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 13:48:15.849088 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.149.195:49171 (local address 127.13.149.254:45043)
0924 13:48:15.849139 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0924 13:48:15.849141 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:48:15.849214 (+ 73us) server_negotiation.cc:407] Connection header received
0924 13:48:15.849264 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:48:15.849265 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:48:15.849276 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:48:15.849323 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:48:15.849637 (+ 314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:48:15.850123 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:48:15.850873 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:48:15.851052 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:48:15.851099 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:48:15.853486 (+ 2387us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.149.195:49171: BlockingRecv error: recv got EOF from 127.13.149.195:49171 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
Sep 24 13:48:16 dist-test-slave-sjl2 krb5kdc[22164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.149.193@KRBTEST.COM for oryx/127.13.149.254@KRBTEST.COM, Server not found in Kerberos database
Sep 24 13:48:16 dist-test-slave-sjl2 krb5kdc[22164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.149.193@KRBTEST.COM for oryx/127.13.149.254@KRBTEST.COM, Server not found in Kerberos database
I20260924 13:48:16.598153 23170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:48:16.593925 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:45043 (local address 127.13.149.193:38985)
0924 13:48:16.594067 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0924 13:48:16.594079 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0924 13:48:16.594128 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:48:16.594330 (+ 202us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:48:16.594331 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:48:16.594486 (+ 155us) client_negotiation.cc:189] Negotiated authn=SASL
0924 13:48:16.594538 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:48:16.594540 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:48:16.595168 (+ 628us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:48:16.595169 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:48:16.595692 (+ 523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:48:16.595694 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:48:16.595991 (+ 297us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:48:16.595991 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:48:16.596029 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:48:16.596480 (+ 451us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 13:48:16.596485 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 13:48:16.598092 (+ 1607us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.149.254:45043: Server oryx/127.13.149.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":57,"threads_started":1}
W20260924 13:48:16.598285 23171 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 13:48:16.593984 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.149.193:38985 (local address 127.13.149.254:45043)
0924 13:48:16.594105 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0924 13:48:16.594107 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:48:16.594117 (+ 10us) server_negotiation.cc:407] Connection header received
0924 13:48:16.594238 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:48:16.594239 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:48:16.594253 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:48:16.594322 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:48:16.594593 (+ 271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:48:16.595113 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:48:16.595745 (+ 632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:48:16.595942 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:48:16.595985 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:48:16.598212 (+ 2227us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.149.193:38985: BlockingRecv error: recv got EOF from 127.13.149.193:38985 (error 108)
Metrics: {"server-negotiator.queue_time_us":102,"thread_start_us":62,"threads_started":1}
Sep 24 13:48:16 dist-test-slave-sjl2 krb5kdc[22164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.149.194@KRBTEST.COM for oryx/127.13.149.254@KRBTEST.COM, Server not found in Kerberos database
Sep 24 13:48:16 dist-test-slave-sjl2 krb5kdc[22164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.149.194@KRBTEST.COM for oryx/127.13.149.254@KRBTEST.COM, Server not found in Kerberos database
I20260924 13:48:16.725558 23172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:48:16.721106 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:45043 (local address 127.13.149.194:48675)
0924 13:48:16.721242 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0924 13:48:16.721253 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0924 13:48:16.721313 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:48:16.721550 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:48:16.721551 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:48:16.721704 (+ 153us) client_negotiation.cc:189] Negotiated authn=SASL
0924 13:48:16.721754 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:48:16.721756 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:48:16.722427 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:48:16.722428 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:48:16.722956 (+ 528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:48:16.722957 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:48:16.723264 (+ 307us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:48:16.723265 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:48:16.723300 (+ 35us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:48:16.723699 (+ 399us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 13:48:16.723703 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0924 13:48:16.725494 (+ 1791us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.149.254:45043: Server oryx/127.13.149.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":69,"threads_started":1}
W20260924 13:48:16.725670 23171 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 13:48:16.721210 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.149.194:48675 (local address 127.13.149.254:45043)
0924 13:48:16.721285 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0924 13:48:16.721287 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:48:16.721300 (+ 13us) server_negotiation.cc:407] Connection header received
0924 13:48:16.721451 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:48:16.721452 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:48:16.721471 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:48:16.721535 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:48:16.721847 (+ 312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:48:16.722365 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:48:16.723016 (+ 651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:48:16.723209 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:48:16.723254 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:48:16.725601 (+ 2347us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.149.194:48675: BlockingRecv error: recv got EOF from 127.13.149.194:48675 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
Sep 24 13:48:16 dist-test-slave-sjl2 krb5kdc[22164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.149.195@KRBTEST.COM for oryx/127.13.149.254@KRBTEST.COM, Server not found in Kerberos database
Sep 24 13:48:16 dist-test-slave-sjl2 krb5kdc[22164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.149.195@KRBTEST.COM for oryx/127.13.149.254@KRBTEST.COM, Server not found in Kerberos database
I20260924 13:48:16.858532 23173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:48:16.854076 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:45043 (local address 127.13.149.195:43807)
0924 13:48:16.854237 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0924 13:48:16.854250 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0924 13:48:16.854319 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:48:16.854465 (+ 146us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:48:16.854465 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:48:16.854639 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0924 13:48:16.854693 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:48:16.854696 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:48:16.855361 (+ 665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:48:16.855362 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:48:16.855993 (+ 631us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:48:16.855996 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:48:16.856316 (+ 320us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:48:16.856317 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:48:16.856368 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:48:16.856768 (+ 400us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 13:48:16.856772 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0924 13:48:16.858472 (+ 1700us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.149.254:45043: Server oryx/127.13.149.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":80,"threads_started":1}
W20260924 13:48:16.858644 23171 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 13:48:16.854214 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.149.195:43807 (local address 127.13.149.254:45043)
0924 13:48:16.854266 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0924 13:48:16.854268 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:48:16.854299 (+ 31us) server_negotiation.cc:407] Connection header received
0924 13:48:16.854388 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:48:16.854389 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:48:16.854402 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:48:16.854450 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:48:16.854757 (+ 307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:48:16.855294 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:48:16.856062 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:48:16.856256 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:48:16.856308 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:48:16.858574 (+ 2266us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.149.195:43807: BlockingRecv error: recv got EOF from 127.13.149.195:43807 (error 108)
Metrics: {"server-negotiator.queue_time_us":25}
Sep 24 13:48:17 dist-test-slave-sjl2 krb5kdc[22164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.149.193@KRBTEST.COM for oryx/127.13.149.254@KRBTEST.COM, Server not found in Kerberos database
Sep 24 13:48:17 dist-test-slave-sjl2 krb5kdc[22164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.149.193@KRBTEST.COM for oryx/127.13.149.254@KRBTEST.COM, Server not found in Kerberos database
I20260924 13:48:17.603082 23174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:48:17.598641 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:45043 (local address 127.13.149.193:53115)
0924 13:48:17.598776 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0924 13:48:17.598788 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0924 13:48:17.598852 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:48:17.599013 (+ 161us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:48:17.599014 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:48:17.599190 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0924 13:48:17.599239 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:48:17.599241 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:48:17.599968 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:48:17.599969 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:48:17.600504 (+ 535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:48:17.600506 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:48:17.600839 (+ 333us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:48:17.600840 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:48:17.600892 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:48:17.601320 (+ 428us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 13:48:17.601325 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 13:48:17.602991 (+ 1666us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.149.254:45043: Server oryx/127.13.149.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":65,"threads_started":1}
W20260924 13:48:17.603157 23175 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 13:48:17.598702 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.149.193:53115 (local address 127.13.149.254:45043)
0924 13:48:17.598796 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0924 13:48:17.598798 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:48:17.598851 (+ 53us) server_negotiation.cc:407] Connection header received
0924 13:48:17.598902 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:48:17.598903 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:48:17.598915 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:48:17.598976 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:48:17.599322 (+ 346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:48:17.599890 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:48:17.600585 (+ 695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:48:17.600775 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:48:17.600827 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:48:17.603097 (+ 2270us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.149.193:53115: BlockingRecv error: recv got EOF from 127.13.149.193:53115 (error 108)
Metrics: {"server-negotiator.queue_time_us":74,"thread_start_us":48,"threads_started":1}
Sep 24 13:48:17 dist-test-slave-sjl2 krb5kdc[22164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.149.194@KRBTEST.COM for oryx/127.13.149.254@KRBTEST.COM, Server not found in Kerberos database
Sep 24 13:48:17 dist-test-slave-sjl2 krb5kdc[22164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.149.194@KRBTEST.COM for oryx/127.13.149.254@KRBTEST.COM, Server not found in Kerberos database
I20260924 13:48:17.730937 23176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:48:17.726117 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:45043 (local address 127.13.149.194:35979)
0924 13:48:17.726269 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0924 13:48:17.726281 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0924 13:48:17.726360 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:48:17.726512 (+ 152us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:48:17.726513 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:48:17.726668 (+ 155us) client_negotiation.cc:189] Negotiated authn=SASL
0924 13:48:17.726719 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:48:17.726721 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:48:17.727444 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:48:17.727446 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:48:17.727990 (+ 544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:48:17.727993 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:48:17.728380 (+ 387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:48:17.728381 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:48:17.728442 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:48:17.728902 (+ 460us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 13:48:17.728907 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 13:48:17.730874 (+ 1967us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.149.254:45043: Server oryx/127.13.149.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":64,"threads_started":1}
W20260924 13:48:17.731009 23175 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 13:48:17.726234 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.149.194:35979 (local address 127.13.149.254:45043)
0924 13:48:17.726298 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0924 13:48:17.726300 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:48:17.726338 (+ 38us) server_negotiation.cc:407] Connection header received
0924 13:48:17.726424 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:48:17.726425 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:48:17.726436 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:48:17.726487 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:48:17.726826 (+ 339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:48:17.727345 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:48:17.728083 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:48:17.728293 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:48:17.728350 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:48:17.730955 (+ 2605us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.149.194:35979: BlockingRecv error: recv got EOF from 127.13.149.194:35979 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
Sep 24 13:48:17 dist-test-slave-sjl2 krb5kdc[22164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.149.195@KRBTEST.COM for oryx/127.13.149.254@KRBTEST.COM, Server not found in Kerberos database
Sep 24 13:48:17 dist-test-slave-sjl2 krb5kdc[22164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.149.195@KRBTEST.COM for oryx/127.13.149.254@KRBTEST.COM, Server not found in Kerberos database
I20260924 13:48:17.863761 23177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:48:17.859111 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:45043 (local address 127.13.149.195:58989)
0924 13:48:17.859260 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0924 13:48:17.859271 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0924 13:48:17.859342 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:48:17.859463 (+ 121us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:48:17.859464 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:48:17.859619 (+ 155us) client_negotiation.cc:189] Negotiated authn=SASL
0924 13:48:17.859665 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:48:17.859667 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:48:17.860308 (+ 641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:48:17.860308 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:48:17.860817 (+ 509us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:48:17.860818 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:48:17.861214 (+ 396us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:48:17.861215 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:48:17.861283 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:48:17.861774 (+ 491us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 13:48:17.861779 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 13:48:17.863697 (+ 1918us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.149.254:45043: Server oryx/127.13.149.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":71,"threads_started":1}
W20260924 13:48:17.863878 23175 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 13:48:17.859214 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.149.195:58989 (local address 127.13.149.254:45043)
0924 13:48:17.859263 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0924 13:48:17.859265 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:48:17.859339 (+ 74us) server_negotiation.cc:407] Connection header received
0924 13:48:17.859388 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:48:17.859389 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:48:17.859400 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:48:17.859446 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:48:17.859736 (+ 290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:48:17.860242 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:48:17.860896 (+ 654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:48:17.861158 (+ 262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:48:17.861214 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:48:17.863814 (+ 2600us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.149.195:58989: BlockingRecv error: recv got EOF from 127.13.149.195:58989 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
Sep 24 13:48:18 dist-test-slave-sjl2 krb5kdc[22164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.149.193@KRBTEST.COM for oryx/127.13.149.254@KRBTEST.COM, Server not found in Kerberos database
Sep 24 13:48:18 dist-test-slave-sjl2 krb5kdc[22164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.149.193@KRBTEST.COM for oryx/127.13.149.254@KRBTEST.COM, Server not found in Kerberos database
I20260924 13:48:18.608013 23178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:48:18.603560 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:45043 (local address 127.13.149.193:52987)
0924 13:48:18.603685 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0924 13:48:18.603695 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 13:48:18.603746 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:48:18.603924 (+ 178us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:48:18.603926 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:48:18.604102 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0924 13:48:18.604154 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:48:18.604156 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:48:18.604812 (+ 656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:48:18.604812 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:48:18.605382 (+ 570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:48:18.605384 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:48:18.605692 (+ 308us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:48:18.605693 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:48:18.605731 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:48:18.606198 (+ 467us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 13:48:18.606203 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 13:48:18.607938 (+ 1735us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.149.254:45043: Server oryx/127.13.149.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":74,"threads_started":1}
W20260924 13:48:18.608134 23179 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 13:48:18.603619 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.149.193:52987 (local address 127.13.149.254:45043)
0924 13:48:18.603722 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0924 13:48:18.603724 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:48:18.603737 (+ 13us) server_negotiation.cc:407] Connection header received
0924 13:48:18.603794 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:48:18.603795 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:48:18.603806 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:48:18.603887 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:48:18.604226 (+ 339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:48:18.604758 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:48:18.605448 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:48:18.605636 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:48:18.605686 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:48:18.608056 (+ 2370us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.149.193:52987: BlockingRecv error: recv got EOF from 127.13.149.193:52987 (error 108)
Metrics: {"server-negotiator.queue_time_us":82,"thread_start_us":53,"threads_started":1}
Sep 24 13:48:18 dist-test-slave-sjl2 krb5kdc[22164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.149.194@KRBTEST.COM for oryx/127.13.149.254@KRBTEST.COM, Server not found in Kerberos database
Sep 24 13:48:18 dist-test-slave-sjl2 krb5kdc[22164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.149.194@KRBTEST.COM for oryx/127.13.149.254@KRBTEST.COM, Server not found in Kerberos database
I20260924 13:48:18.739174 23180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:48:18.731573 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:45043 (local address 127.13.149.194:41251)
0924 13:48:18.731702 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0924 13:48:18.731715 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0924 13:48:18.731786 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:48:18.731913 (+ 127us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:48:18.731914 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:48:18.732075 (+ 161us) client_negotiation.cc:189] Negotiated authn=SASL
0924 13:48:18.732124 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:48:18.732126 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:48:18.732870 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:48:18.732871 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:48:18.733415 (+ 544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:48:18.733416 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:48:18.733723 (+ 307us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:48:18.733724 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:48:18.733762 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:48:18.737320 (+ 3558us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 13:48:18.737327 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0924 13:48:18.739112 (+ 1785us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.149.254:45043: Server oryx/127.13.149.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":58,"threads_started":1}
W20260924 13:48:18.739275 23179 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 13:48:18.731679 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.149.194:41251 (local address 127.13.149.254:45043)
0924 13:48:18.731759 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0924 13:48:18.731761 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:48:18.731772 (+ 11us) server_negotiation.cc:407] Connection header received
0924 13:48:18.731837 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:48:18.731837 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:48:18.731847 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:48:18.731902 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:48:18.732207 (+ 305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:48:18.732796 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:48:18.733486 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:48:18.733670 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:48:18.733717 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:48:18.739216 (+ 5499us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.149.194:41251: BlockingRecv error: recv got EOF from 127.13.149.194:41251 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
Sep 24 13:48:18 dist-test-slave-sjl2 krb5kdc[22164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.149.195@KRBTEST.COM for oryx/127.13.149.254@KRBTEST.COM, Server not found in Kerberos database
Sep 24 13:48:18 dist-test-slave-sjl2 krb5kdc[22164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.149.195@KRBTEST.COM for oryx/127.13.149.254@KRBTEST.COM, Server not found in Kerberos database
I20260924 13:48:18.868755 23181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:48:18.864292 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:45043 (local address 127.13.149.195:38615)
0924 13:48:18.864454 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0924 13:48:18.864466 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0924 13:48:18.864561 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:48:18.864702 (+ 141us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:48:18.864703 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:48:18.864893 (+ 190us) client_negotiation.cc:189] Negotiated authn=SASL
0924 13:48:18.864971 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:48:18.864974 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:48:18.865624 (+ 650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:48:18.865626 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:48:18.866139 (+ 513us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:48:18.866141 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:48:18.866482 (+ 341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:48:18.866483 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:48:18.866543 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:48:18.866979 (+ 436us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 13:48:18.866984 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 13:48:18.868686 (+ 1702us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.149.254:45043: Server oryx/127.13.149.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":97,"threads_started":1}
W20260924 13:48:18.868852 23179 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 13:48:18.864394 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.149.195:38615 (local address 127.13.149.254:45043)
0924 13:48:18.864460 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0924 13:48:18.864462 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:48:18.864541 (+ 79us) server_negotiation.cc:407] Connection header received
0924 13:48:18.864629 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:48:18.864630 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:48:18.864641 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:48:18.864688 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:48:18.865062 (+ 374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:48:18.865548 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:48:18.866224 (+ 676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:48:18.866403 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:48:18.866455 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:48:18.868782 (+ 2327us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.149.195:38615: BlockingRecv error: recv got EOF from 127.13.149.195:38615 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
Sep 24 13:48:19 dist-test-slave-sjl2 krb5kdc[22164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.149.193@KRBTEST.COM for oryx/127.13.149.254@KRBTEST.COM, Server not found in Kerberos database
Sep 24 13:48:19 dist-test-slave-sjl2 krb5kdc[22164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.149.193@KRBTEST.COM for oryx/127.13.149.254@KRBTEST.COM, Server not found in Kerberos database
I20260924 13:48:19.612810 23182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:48:19.608561 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:45043 (local address 127.13.149.193:57565)
0924 13:48:19.608688 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0924 13:48:19.608700 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0924 13:48:19.608745 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:48:19.608866 (+ 121us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:48:19.608867 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:48:19.609035 (+ 168us) client_negotiation.cc:189] Negotiated authn=SASL
0924 13:48:19.609081 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:48:19.609083 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:48:19.609713 (+ 630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:48:19.609713 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:48:19.610229 (+ 516us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:48:19.610230 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:48:19.610599 (+ 369us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:48:19.610600 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:48:19.610654 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:48:19.611102 (+ 448us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 13:48:19.611107 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 13:48:19.612751 (+ 1644us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.149.254:45043: Server oryx/127.13.149.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":63,"threads_started":1}
W20260924 13:48:19.612980 23183 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 13:48:19.608614 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.149.193:57565 (local address 127.13.149.254:45043)
0924 13:48:19.608718 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0924 13:48:19.608720 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:48:19.608730 (+ 10us) server_negotiation.cc:407] Connection header received
0924 13:48:19.608786 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:48:19.608787 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:48:19.608796 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:48:19.608858 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:48:19.609130 (+ 272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:48:19.609654 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:48:19.610297 (+ 643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:48:19.610515 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:48:19.610568 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:48:19.612857 (+ 2289us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.149.193:57565: BlockingRecv error: recv got EOF from 127.13.149.193:57565 (error 108)
Metrics: {"server-negotiator.queue_time_us":85,"thread_start_us":61,"threads_started":1}
Sep 24 13:48:19 dist-test-slave-sjl2 krb5kdc[22164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.149.194@KRBTEST.COM for oryx/127.13.149.254@KRBTEST.COM, Server not found in Kerberos database
Sep 24 13:48:19 dist-test-slave-sjl2 krb5kdc[22164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.149.194@KRBTEST.COM for oryx/127.13.149.254@KRBTEST.COM, Server not found in Kerberos database
I20260924 13:48:19.744617 23184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:48:19.739699 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:45043 (local address 127.13.149.194:55035)
0924 13:48:19.739893 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0924 13:48:19.739903 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 13:48:19.739967 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:48:19.740113 (+ 146us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:48:19.740114 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:48:19.740321 (+ 207us) client_negotiation.cc:189] Negotiated authn=SASL
0924 13:48:19.740379 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:48:19.740381 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:48:19.741082 (+ 701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:48:19.741083 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:48:19.741647 (+ 564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:48:19.741649 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:48:19.742039 (+ 390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:48:19.742040 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:48:19.742099 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:48:19.742589 (+ 490us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 13:48:19.742594 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 13:48:19.744536 (+ 1942us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.149.254:45043: Server oryx/127.13.149.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":75,"threads_started":1}
W20260924 13:48:19.744704 23183 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 13:48:19.739818 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.149.194:55035 (local address 127.13.149.254:45043)
0924 13:48:19.739884 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0924 13:48:19.739886 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:48:19.739959 (+ 73us) server_negotiation.cc:407] Connection header received
0924 13:48:19.740035 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:48:19.740036 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:48:19.740047 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:48:19.740104 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:48:19.740443 (+ 339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:48:19.741014 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:48:19.741727 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:48:19.741967 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:48:19.742021 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:48:19.744651 (+ 2630us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.149.194:55035: BlockingRecv error: recv got EOF from 127.13.149.194:55035 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
Sep 24 13:48:19 dist-test-slave-sjl2 krb5kdc[22164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.149.195@KRBTEST.COM for oryx/127.13.149.254@KRBTEST.COM, Server not found in Kerberos database
Sep 24 13:48:19 dist-test-slave-sjl2 krb5kdc[22164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.149.195@KRBTEST.COM for oryx/127.13.149.254@KRBTEST.COM, Server not found in Kerberos database
I20260924 13:48:19.873752 23185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:48:19.869291 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:45043 (local address 127.13.149.195:34985)
0924 13:48:19.869462 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0924 13:48:19.869475 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0924 13:48:19.869544 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:48:19.869662 (+ 118us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:48:19.869662 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:48:19.869811 (+ 149us) client_negotiation.cc:189] Negotiated authn=SASL
0924 13:48:19.869868 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:48:19.869870 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:48:19.870512 (+ 642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:48:19.870513 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:48:19.871029 (+ 516us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:48:19.871030 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:48:19.871397 (+ 367us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:48:19.871398 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:48:19.871448 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:48:19.872012 (+ 564us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 13:48:19.872017 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 13:48:19.873693 (+ 1676us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.149.254:45043: Server oryx/127.13.149.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":75,"threads_started":1}
W20260924 13:48:19.873838 23183 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 13:48:19.869459 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.149.195:34985 (local address 127.13.149.254:45043)
0924 13:48:19.869523 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0924 13:48:19.869524 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0924 13:48:19.869534 (+ 10us) server_negotiation.cc:407] Connection header received
0924 13:48:19.869593 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:48:19.869594 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:48:19.869606 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:48:19.869649 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:48:19.869921 (+ 272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:48:19.870410 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:48:19.871107 (+ 697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:48:19.871292 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:48:19.871344 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:48:19.873779 (+ 2435us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.149.195:34985: BlockingRecv error: recv got EOF from 127.13.149.195:34985 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
Sep 24 13:48:20 dist-test-slave-sjl2 krb5kdc[22164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.149.193@KRBTEST.COM for oryx/127.13.149.254@KRBTEST.COM, Server not found in Kerberos database
Sep 24 13:48:20 dist-test-slave-sjl2 krb5kdc[22164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.149.193@KRBTEST.COM for oryx/127.13.149.254@KRBTEST.COM, Server not found in Kerberos database
I20260924 13:48:20.617877 23186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:48:20.613313 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:45043 (local address 127.13.149.193:55997)
0924 13:48:20.613457 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0924 13:48:20.613471 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0924 13:48:20.613551 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:48:20.613720 (+ 169us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:48:20.613721 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:48:20.613889 (+ 168us) client_negotiation.cc:189] Negotiated authn=SASL
0924 13:48:20.613941 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:48:20.613943 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:48:20.614599 (+ 656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:48:20.614600 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:48:20.615124 (+ 524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:48:20.615126 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:48:20.615445 (+ 319us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:48:20.615446 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:48:20.615493 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:48:20.615906 (+ 413us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 13:48:20.615911 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 13:48:20.617798 (+ 1887us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.149.254:45043: Server oryx/127.13.149.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":66,"threads_started":1}
W20260924 13:48:20.618001 23187 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 13:48:20.613398 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.149.193:55997 (local address 127.13.149.254:45043)
0924 13:48:20.613499 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0924 13:48:20.613501 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:48:20.613540 (+ 39us) server_negotiation.cc:407] Connection header received
0924 13:48:20.613610 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:48:20.613611 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:48:20.613622 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:48:20.613685 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:48:20.614001 (+ 316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:48:20.614501 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:48:20.615190 (+ 689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:48:20.615382 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:48:20.615429 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:48:20.617934 (+ 2505us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.149.193:55997: BlockingRecv error: recv got EOF from 127.13.149.193:55997 (error 108)
Metrics: {"server-negotiator.queue_time_us":83,"thread_start_us":46,"threads_started":1}
Sep 24 13:48:20 dist-test-slave-sjl2 krb5kdc[22164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.149.194@KRBTEST.COM for oryx/127.13.149.254@KRBTEST.COM, Server not found in Kerberos database
Sep 24 13:48:20 dist-test-slave-sjl2 krb5kdc[22164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.149.194@KRBTEST.COM for oryx/127.13.149.254@KRBTEST.COM, Server not found in Kerberos database
I20260924 13:48:20.749732 23188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:48:20.745185 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:45043 (local address 127.13.149.194:42633)
0924 13:48:20.745307 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0924 13:48:20.745318 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0924 13:48:20.745380 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:48:20.745529 (+ 149us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:48:20.745530 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:48:20.745714 (+ 184us) client_negotiation.cc:189] Negotiated authn=SASL
0924 13:48:20.745762 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:48:20.745764 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:48:20.746434 (+ 670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:48:20.746435 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:48:20.746961 (+ 526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:48:20.746963 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:48:20.747275 (+ 312us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:48:20.747276 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:48:20.747316 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:48:20.747736 (+ 420us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 13:48:20.747740 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0924 13:48:20.749649 (+ 1909us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.149.254:45043: Server oryx/127.13.149.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":64,"threads_started":1}
W20260924 13:48:20.749820 23187 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 13:48:20.745280 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.149.194:42633 (local address 127.13.149.254:45043)
0924 13:48:20.745341 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0924 13:48:20.745349 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0924 13:48:20.745360 (+ 11us) server_negotiation.cc:407] Connection header received
0924 13:48:20.745430 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:48:20.745431 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:48:20.745441 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:48:20.745490 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:48:20.745841 (+ 351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:48:20.746336 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:48:20.747033 (+ 697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:48:20.747220 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:48:20.747270 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:48:20.749762 (+ 2492us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.149.194:42633: BlockingRecv error: recv got EOF from 127.13.149.194:42633 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
Sep 24 13:48:20 dist-test-slave-sjl2 krb5kdc[22164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.149.195@KRBTEST.COM for oryx/127.13.149.254@KRBTEST.COM, Server not found in Kerberos database
Sep 24 13:48:20 dist-test-slave-sjl2 krb5kdc[22164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.149.195@KRBTEST.COM for oryx/127.13.149.254@KRBTEST.COM, Server not found in Kerberos database
I20260924 13:48:20.878664 23189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:48:20.874251 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:45043 (local address 127.13.149.195:33553)
0924 13:48:20.874366 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0924 13:48:20.874376 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 13:48:20.874426 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:48:20.874578 (+ 152us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:48:20.874579 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:48:20.874738 (+ 159us) client_negotiation.cc:189] Negotiated authn=SASL
0924 13:48:20.874783 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:48:20.874785 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:48:20.875419 (+ 634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:48:20.875420 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:48:20.875968 (+ 548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:48:20.875971 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:48:20.876294 (+ 323us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:48:20.876295 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:48:20.876350 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:48:20.876784 (+ 434us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 13:48:20.876788 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0924 13:48:20.878603 (+ 1815us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.149.254:45043: Server oryx/127.13.149.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":96,"thread_start_us":63,"threads_started":1}
W20260924 13:48:20.878772 23187 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 13:48:20.874377 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.149.195:33553 (local address 127.13.149.254:45043)
0924 13:48:20.874471 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0924 13:48:20.874473 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:48:20.874481 (+ 8us) server_negotiation.cc:407] Connection header received
0924 13:48:20.874505 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:48:20.874506 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:48:20.874517 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:48:20.874561 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:48:20.874849 (+ 288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:48:20.875341 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:48:20.876049 (+ 708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:48:20.876234 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:48:20.876289 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:48:20.878700 (+ 2411us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.149.195:33553: BlockingRecv error: recv got EOF from 127.13.149.195:33553 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
Sep 24 13:48:21 dist-test-slave-sjl2 krb5kdc[22164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.149.193@KRBTEST.COM for oryx/127.13.149.254@KRBTEST.COM, Server not found in Kerberos database
Sep 24 13:48:21 dist-test-slave-sjl2 krb5kdc[22164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.149.193@KRBTEST.COM for oryx/127.13.149.254@KRBTEST.COM, Server not found in Kerberos database
I20260924 13:48:21.623178 23190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:48:21.618531 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:45043 (local address 127.13.149.193:52293)
0924 13:48:21.618703 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0924 13:48:21.618716 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0924 13:48:21.618787 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:48:21.618914 (+ 127us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:48:21.618914 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:48:21.619093 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0924 13:48:21.619147 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:48:21.619148 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:48:21.619934 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:48:21.619935 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:48:21.620622 (+ 687us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:48:21.620625 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:48:21.621019 (+ 394us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:48:21.621024 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:48:21.621070 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:48:21.621481 (+ 411us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 13:48:21.621486 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 13:48:21.623118 (+ 1632us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.149.254:45043: Server oryx/127.13.149.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":70,"threads_started":1}
W20260924 13:48:21.623282 23191 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 13:48:21.618650 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.149.193:52293 (local address 127.13.149.254:45043)
0924 13:48:21.618779 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0924 13:48:21.618781 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:48:21.618791 (+ 10us) server_negotiation.cc:407] Connection header received
0924 13:48:21.618832 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:48:21.618833 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:48:21.618843 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:48:21.618907 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:48:21.619247 (+ 340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:48:21.619795 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:48:21.620700 (+ 905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:48:21.620906 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:48:21.620998 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:48:21.623214 (+ 2216us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.149.193:52293: BlockingRecv error: recv got EOF from 127.13.149.193:52293 (error 108)
Metrics: {"server-negotiator.queue_time_us":109,"thread_start_us":68,"threads_started":1}
Sep 24 13:48:21 dist-test-slave-sjl2 krb5kdc[22164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.149.194@KRBTEST.COM for oryx/127.13.149.254@KRBTEST.COM, Server not found in Kerberos database
Sep 24 13:48:21 dist-test-slave-sjl2 krb5kdc[22164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.149.194@KRBTEST.COM for oryx/127.13.149.254@KRBTEST.COM, Server not found in Kerberos database
I20260924 13:48:21.754652 23192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:48:21.750275 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:45043 (local address 127.13.149.194:60467)
0924 13:48:21.750450 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0924 13:48:21.750461 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0924 13:48:21.750541 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:48:21.750693 (+ 152us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:48:21.750694 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:48:21.750894 (+ 200us) client_negotiation.cc:189] Negotiated authn=SASL
0924 13:48:21.750943 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:48:21.750945 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:48:21.751581 (+ 636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:48:21.751582 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:48:21.752123 (+ 541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:48:21.752124 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:48:21.752413 (+ 289us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:48:21.752413 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:48:21.752457 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:48:21.752859 (+ 402us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 13:48:21.752864 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 13:48:21.754592 (+ 1728us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.149.254:45043: Server oryx/127.13.149.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":75,"threads_started":1}
W20260924 13:48:21.754812 23191 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 13:48:21.750379 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.149.194:60467 (local address 127.13.149.254:45043)
0924 13:48:21.750450 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0924 13:48:21.750452 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:48:21.750521 (+ 69us) server_negotiation.cc:407] Connection header received
0924 13:48:21.750610 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:48:21.750611 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:48:21.750625 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:48:21.750677 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:48:21.751005 (+ 328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:48:21.751496 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:48:21.752190 (+ 694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:48:21.752365 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:48:21.752407 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:48:21.754703 (+ 2296us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.149.194:60467: BlockingRecv error: recv got EOF from 127.13.149.194:60467 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
Sep 24 13:48:21 dist-test-slave-sjl2 krb5kdc[22164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.149.195@KRBTEST.COM for oryx/127.13.149.254@KRBTEST.COM, Server not found in Kerberos database
Sep 24 13:48:21 dist-test-slave-sjl2 krb5kdc[22164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.149.195@KRBTEST.COM for oryx/127.13.149.254@KRBTEST.COM, Server not found in Kerberos database
I20260924 13:48:21.883535 23193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:48:21.879204 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:45043 (local address 127.13.149.195:57979)
0924 13:48:21.879345 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0924 13:48:21.879356 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0924 13:48:21.879431 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:48:21.879561 (+ 130us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:48:21.879562 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:48:21.879732 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0924 13:48:21.879776 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:48:21.879778 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:48:21.880470 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:48:21.880471 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:48:21.881023 (+ 552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:48:21.881025 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:48:21.881357 (+ 332us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:48:21.881358 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:48:21.881413 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:48:21.881821 (+ 408us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 13:48:21.881825 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0924 13:48:21.883457 (+ 1632us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.149.254:45043: Server oryx/127.13.149.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":78,"threads_started":1}
W20260924 13:48:21.883617 23191 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 13:48:21.879341 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.149.195:57979 (local address 127.13.149.254:45043)
0924 13:48:21.879422 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0924 13:48:21.879424 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:48:21.879435 (+ 11us) server_negotiation.cc:407] Connection header received
0924 13:48:21.879486 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:48:21.879487 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:48:21.879500 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:48:21.879554 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:48:21.879848 (+ 294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:48:21.880402 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:48:21.881108 (+ 706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:48:21.881298 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:48:21.881352 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:48:21.883565 (+ 2213us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.149.195:57979: BlockingRecv error: recv got EOF from 127.13.149.195:57979 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Sep 24 13:48:22 dist-test-slave-sjl2 krb5kdc[22164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.149.193@KRBTEST.COM for oryx/127.13.149.254@KRBTEST.COM, Server not found in Kerberos database
Sep 24 13:48:22 dist-test-slave-sjl2 krb5kdc[22164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.149.193@KRBTEST.COM for oryx/127.13.149.254@KRBTEST.COM, Server not found in Kerberos database
I20260924 13:48:22.628882 23194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:48:22.623750 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:45043 (local address 127.13.149.193:47833)
0924 13:48:22.623947 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0924 13:48:22.623962 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0924 13:48:22.624043 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:48:22.624329 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:48:22.624331 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:48:22.624491 (+ 160us) client_negotiation.cc:189] Negotiated authn=SASL
0924 13:48:22.624549 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:48:22.624551 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:48:22.625247 (+ 696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:48:22.625248 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:48:22.625812 (+ 564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:48:22.625814 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:48:22.626171 (+ 357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:48:22.626171 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:48:22.626211 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:48:22.626688 (+ 477us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 13:48:22.626693 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 13:48:22.628818 (+ 2125us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.149.254:45043: Server oryx/127.13.149.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":76,"threads_started":1}
W20260924 13:48:22.629020 23195 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 13:48:22.623942 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.149.193:47833 (local address 127.13.149.254:45043)
0924 13:48:22.624125 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0924 13:48:22.624127 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:48:22.624136 (+ 9us) server_negotiation.cc:407] Connection header received
0924 13:48:22.624163 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:48:22.624163 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:48:22.624196 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:48:22.624264 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:48:22.624621 (+ 357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:48:22.625157 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:48:22.625908 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:48:22.626110 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:48:22.626165 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:48:22.628946 (+ 2781us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.149.193:47833: BlockingRecv error: recv got EOF from 127.13.149.193:47833 (error 108)
Metrics: {"server-negotiator.queue_time_us":160,"thread_start_us":97,"threads_started":1}
Sep 24 13:48:22 dist-test-slave-sjl2 krb5kdc[22164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.149.194@KRBTEST.COM for oryx/127.13.149.254@KRBTEST.COM, Server not found in Kerberos database
Sep 24 13:48:22 dist-test-slave-sjl2 krb5kdc[22164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.149.194@KRBTEST.COM for oryx/127.13.149.254@KRBTEST.COM, Server not found in Kerberos database
I20260924 13:48:22.760572 23196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:48:22.755279 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:45043 (local address 127.13.149.194:42649)
0924 13:48:22.755501 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0924 13:48:22.755513 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0924 13:48:22.755580 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:48:22.755770 (+ 190us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:48:22.755772 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:48:22.755935 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0924 13:48:22.755986 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:48:22.755987 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:48:22.756866 (+ 879us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:48:22.756868 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:48:22.757468 (+ 600us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:48:22.757471 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:48:22.757915 (+ 444us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:48:22.757917 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:48:22.757970 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:48:22.758462 (+ 492us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 13:48:22.758467 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 13:48:22.760424 (+ 1957us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.149.254:45043: Server oryx/127.13.149.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":203,"thread_start_us":149,"threads_started":1}
W20260924 13:48:22.760665 23195 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 13:48:22.755424 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.149.194:42649 (local address 127.13.149.254:45043)
0924 13:48:22.755582 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0924 13:48:22.755584 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:48:22.755618 (+ 34us) server_negotiation.cc:407] Connection header received
0924 13:48:22.755648 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:48:22.755649 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:48:22.755660 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:48:22.755711 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:48:22.756070 (+ 359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:48:22.756766 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:48:22.757584 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:48:22.757794 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:48:22.757854 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:48:22.760609 (+ 2755us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.149.194:42649: BlockingRecv error: recv got EOF from 127.13.149.194:42649 (error 108)
Metrics: {"server-negotiator.queue_time_us":127}
Sep 24 13:48:22 dist-test-slave-sjl2 krb5kdc[22164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.149.195@KRBTEST.COM for oryx/127.13.149.254@KRBTEST.COM, Server not found in Kerberos database
Sep 24 13:48:22 dist-test-slave-sjl2 krb5kdc[22164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.149.195@KRBTEST.COM for oryx/127.13.149.254@KRBTEST.COM, Server not found in Kerberos database
I20260924 13:48:22.888856 23197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:48:22.884079 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:45043 (local address 127.13.149.195:51455)
0924 13:48:22.884197 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0924 13:48:22.884208 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0924 13:48:22.884261 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:48:22.884426 (+ 165us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:48:22.884427 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:48:22.884579 (+ 152us) client_negotiation.cc:189] Negotiated authn=SASL
0924 13:48:22.884628 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:48:22.884629 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:48:22.885398 (+ 769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:48:22.885399 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:48:22.885952 (+ 553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:48:22.885954 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:48:22.886308 (+ 354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:48:22.886309 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:48:22.886352 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:48:22.886850 (+ 498us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 13:48:22.886855 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 13:48:22.888780 (+ 1925us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.149.254:45043: Server oryx/127.13.149.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":65,"threads_started":1}
W20260924 13:48:22.889022 23195 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 13:48:22.884201 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.149.195:51455 (local address 127.13.149.254:45043)
0924 13:48:22.884292 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0924 13:48:22.884294 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:48:22.884306 (+ 12us) server_negotiation.cc:407] Connection header received
0924 13:48:22.884333 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:48:22.884334 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:48:22.884347 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:48:22.884415 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:48:22.884692 (+ 277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:48:22.885282 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:48:22.886048 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:48:22.886241 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:48:22.886304 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:48:22.888958 (+ 2654us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.149.195:51455: BlockingRecv error: recv got EOF from 127.13.149.195:51455 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
Sep 24 13:48:23 dist-test-slave-sjl2 krb5kdc[22164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.149.193@KRBTEST.COM for oryx/127.13.149.254@KRBTEST.COM, Server not found in Kerberos database
Sep 24 13:48:23 dist-test-slave-sjl2 krb5kdc[22164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.149.193@KRBTEST.COM for oryx/127.13.149.254@KRBTEST.COM, Server not found in Kerberos database
I20260924 13:48:23.634128 23198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:48:23.629385 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:45043 (local address 127.13.149.193:46047)
0924 13:48:23.629540 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0924 13:48:23.629550 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 13:48:23.629599 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:48:23.629729 (+ 130us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:48:23.629730 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:48:23.629927 (+ 197us) client_negotiation.cc:189] Negotiated authn=SASL
0924 13:48:23.629976 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:48:23.629977 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:48:23.630644 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:48:23.630645 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:48:23.631182 (+ 537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:48:23.631184 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:48:23.631585 (+ 401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:48:23.631586 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:48:23.631639 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:48:23.632071 (+ 432us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 13:48:23.632075 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0924 13:48:23.634024 (+ 1949us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.149.254:45043: Server oryx/127.13.149.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":86,"threads_started":1}
W20260924 13:48:23.634198 23199 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 13:48:23.629471 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.149.193:46047 (local address 127.13.149.254:45043)
0924 13:48:23.629607 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0924 13:48:23.629609 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:48:23.629617 (+ 8us) server_negotiation.cc:407] Connection header received
0924 13:48:23.629642 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:48:23.629642 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:48:23.629653 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:48:23.629714 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:48:23.630044 (+ 330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:48:23.630550 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:48:23.631276 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:48:23.631485 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:48:23.631541 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:48:23.634135 (+ 2594us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.149.193:46047: BlockingRecv error: recv got EOF from 127.13.149.193:46047 (error 108)
Metrics: {"server-negotiator.queue_time_us":118,"thread_start_us":68,"threads_started":1}
Sep 24 13:48:23 dist-test-slave-sjl2 krb5kdc[22164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.149.194@KRBTEST.COM for oryx/127.13.149.254@KRBTEST.COM, Server not found in Kerberos database
Sep 24 13:48:23 dist-test-slave-sjl2 krb5kdc[22164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.149.194@KRBTEST.COM for oryx/127.13.149.254@KRBTEST.COM, Server not found in Kerberos database
I20260924 13:48:23.771836 23200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:48:23.761214 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:45043 (local address 127.13.149.194:54409)
0924 13:48:23.761357 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0924 13:48:23.761367 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 13:48:23.761415 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:48:23.761613 (+ 198us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:48:23.761614 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:48:23.761776 (+ 162us) client_negotiation.cc:189] Negotiated authn=SASL
0924 13:48:23.761823 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:48:23.761825 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:48:23.762560 (+ 735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:48:23.762561 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:48:23.763132 (+ 571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:48:23.763133 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:48:23.763543 (+ 410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:48:23.763544 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:48:23.763598 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:48:23.769925 (+ 6327us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 13:48:23.769933 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0924 13:48:23.771757 (+ 1824us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.149.254:45043: Server oryx/127.13.149.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":79,"threads_started":1}
W20260924 13:48:23.771925 23199 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 13:48:23.761377 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.149.194:54409 (local address 127.13.149.254:45043)
0924 13:48:23.761452 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0924 13:48:23.761454 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:48:23.761463 (+ 9us) server_negotiation.cc:407] Connection header received
0924 13:48:23.761499 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:48:23.761500 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:48:23.761510 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:48:23.761574 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:48:23.761944 (+ 370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:48:23.762463 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:48:23.763227 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:48:23.763448 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:48:23.763514 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:48:23.771868 (+ 8354us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.149.194:54409: BlockingRecv error: recv got EOF from 127.13.149.194:54409 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
Sep 24 13:48:23 dist-test-slave-sjl2 krb5kdc[22164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.149.195@KRBTEST.COM for oryx/127.13.149.254@KRBTEST.COM, Server not found in Kerberos database
Sep 24 13:48:23 dist-test-slave-sjl2 krb5kdc[22164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.149.195@KRBTEST.COM for oryx/127.13.149.254@KRBTEST.COM, Server not found in Kerberos database
I20260924 13:48:23.894197 23201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:48:23.889442 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:45043 (local address 127.13.149.195:34331)
0924 13:48:23.889566 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0924 13:48:23.889579 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0924 13:48:23.889651 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:48:23.889829 (+ 178us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:48:23.889830 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:48:23.889999 (+ 169us) client_negotiation.cc:189] Negotiated authn=SASL
0924 13:48:23.890047 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:48:23.890049 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:48:23.890853 (+ 804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:48:23.890854 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:48:23.891366 (+ 512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:48:23.891368 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:48:23.891746 (+ 378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:48:23.891747 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:48:23.891785 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:48:23.892207 (+ 422us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 13:48:23.892212 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 13:48:23.894139 (+ 1927us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.149.254:45043: Server oryx/127.13.149.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":58,"threads_started":1}
W20260924 13:48:23.894299 23199 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 13:48:23.889557 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.149.195:34331 (local address 127.13.149.254:45043)
0924 13:48:23.889644 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0924 13:48:23.889646 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:48:23.889658 (+ 12us) server_negotiation.cc:407] Connection header received
0924 13:48:23.889708 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:48:23.889709 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:48:23.889721 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:48:23.889769 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:48:23.890112 (+ 343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:48:23.890790 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:48:23.891499 (+ 709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:48:23.891682 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:48:23.891737 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:48:23.894236 (+ 2499us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.149.195:34331: BlockingRecv error: recv got EOF from 127.13.149.195:34331 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Sep 24 13:48:24 dist-test-slave-sjl2 krb5kdc[22164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.149.193@KRBTEST.COM for oryx/127.13.149.254@KRBTEST.COM, Server not found in Kerberos database
Sep 24 13:48:24 dist-test-slave-sjl2 krb5kdc[22164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.149.193@KRBTEST.COM for oryx/127.13.149.254@KRBTEST.COM, Server not found in Kerberos database
I20260924 13:48:24.638970 23202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:48:24.634627 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:45043 (local address 127.13.149.193:40959)
0924 13:48:24.634745 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0924 13:48:24.634758 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0924 13:48:24.634816 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:48:24.634958 (+ 142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:48:24.634958 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:48:24.635115 (+ 157us) client_negotiation.cc:189] Negotiated authn=SASL
0924 13:48:24.635164 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:48:24.635166 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:48:24.635879 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:48:24.635880 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:48:24.636424 (+ 544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:48:24.636425 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:48:24.636765 (+ 340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:48:24.636766 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:48:24.636806 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:48:24.637313 (+ 507us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 13:48:24.637317 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0924 13:48:24.638910 (+ 1593us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.149.254:45043: Server oryx/127.13.149.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":54,"threads_started":1}
W20260924 13:48:24.639102 23203 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 13:48:24.634695 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.149.193:40959 (local address 127.13.149.254:45043)
0924 13:48:24.634828 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0924 13:48:24.634830 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:48:24.634865 (+ 35us) server_negotiation.cc:407] Connection header received
0924 13:48:24.634889 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:48:24.634890 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:48:24.634900 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:48:24.634948 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:48:24.635248 (+ 300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:48:24.635810 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:48:24.636510 (+ 700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:48:24.636704 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:48:24.636763 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:48:24.639024 (+ 2261us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.149.193:40959: BlockingRecv error: recv got EOF from 127.13.149.193:40959 (error 108)
Metrics: {"server-negotiator.queue_time_us":113,"thread_start_us":62,"threads_started":1}
Sep 24 13:48:24 dist-test-slave-sjl2 krb5kdc[22164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.149.194@KRBTEST.COM for oryx/127.13.149.254@KRBTEST.COM, Server not found in Kerberos database
Sep 24 13:48:24 dist-test-slave-sjl2 krb5kdc[22164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.149.194@KRBTEST.COM for oryx/127.13.149.254@KRBTEST.COM, Server not found in Kerberos database
I20260924 13:48:24.776753 23204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:48:24.772359 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:45043 (local address 127.13.149.194:44753)
0924 13:48:24.772493 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0924 13:48:24.772504 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0924 13:48:24.772571 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:48:24.772702 (+ 131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:48:24.772703 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:48:24.772883 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0924 13:48:24.772957 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:48:24.772959 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:48:24.773609 (+ 650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:48:24.773610 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:48:24.774155 (+ 545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:48:24.774157 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:48:24.774474 (+ 317us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:48:24.774475 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:48:24.774528 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:48:24.774971 (+ 443us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 13:48:24.774976 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 13:48:24.776690 (+ 1714us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.149.254:45043: Server oryx/127.13.149.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":67,"threads_started":1}
W20260924 13:48:24.776935 23203 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 13:48:24.772473 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.149.194:44753 (local address 127.13.149.254:45043)
0924 13:48:24.772542 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0924 13:48:24.772544 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:48:24.772556 (+ 12us) server_negotiation.cc:407] Connection header received
0924 13:48:24.772638 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:48:24.772639 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:48:24.772648 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:48:24.772694 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:48:24.773027 (+ 333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:48:24.773537 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:48:24.774227 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:48:24.774416 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:48:24.774470 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:48:24.776847 (+ 2377us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.149.194:44753: BlockingRecv error: recv got EOF from 127.13.149.194:44753 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
Sep 24 13:48:24 dist-test-slave-sjl2 krb5kdc[22164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.149.195@KRBTEST.COM for oryx/127.13.149.254@KRBTEST.COM, Server not found in Kerberos database
Sep 24 13:48:24 dist-test-slave-sjl2 krb5kdc[22164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.149.195@KRBTEST.COM for oryx/127.13.149.254@KRBTEST.COM, Server not found in Kerberos database
I20260924 13:48:24.899169 23205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:48:24.894669 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:45043 (local address 127.13.149.195:50331)
0924 13:48:24.894821 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0924 13:48:24.894834 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0924 13:48:24.894911 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:48:24.895057 (+ 146us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:48:24.895059 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:48:24.895230 (+ 171us) client_negotiation.cc:189] Negotiated authn=SASL
0924 13:48:24.895277 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:48:24.895278 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:48:24.895920 (+ 642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:48:24.895921 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:48:24.896443 (+ 522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:48:24.896445 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:48:24.896764 (+ 319us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:48:24.896765 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:48:24.896820 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:48:24.897311 (+ 491us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 13:48:24.897315 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0924 13:48:24.899080 (+ 1765us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.149.254:45043: Server oryx/127.13.149.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":55,"threads_started":1}
W20260924 13:48:24.899246 23203 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 13:48:24.894791 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.149.195:50331 (local address 127.13.149.254:45043)
0924 13:48:24.894836 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0924 13:48:24.894838 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:48:24.894892 (+ 54us) server_negotiation.cc:407] Connection header received
0924 13:48:24.894985 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:48:24.894986 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:48:24.895000 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:48:24.895042 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:48:24.895346 (+ 304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:48:24.895846 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:48:24.896520 (+ 674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:48:24.896703 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:48:24.896757 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:48:24.899190 (+ 2433us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.149.195:50331: BlockingRecv error: recv got EOF from 127.13.149.195:50331 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
Sep 24 13:48:25 dist-test-slave-sjl2 krb5kdc[22164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.149.193@KRBTEST.COM for oryx/127.13.149.254@KRBTEST.COM, Server not found in Kerberos database
Sep 24 13:48:25 dist-test-slave-sjl2 krb5kdc[22164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.149.193@KRBTEST.COM for oryx/127.13.149.254@KRBTEST.COM, Server not found in Kerberos database
I20260924 13:48:25.644002 23206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:48:25.639472 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:45043 (local address 127.13.149.193:57613)
0924 13:48:25.639602 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0924 13:48:25.639612 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 13:48:25.639659 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:48:25.639805 (+ 146us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:48:25.639806 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:48:25.639973 (+ 167us) client_negotiation.cc:189] Negotiated authn=SASL
0924 13:48:25.640020 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:48:25.640021 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:48:25.640630 (+ 609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:48:25.640630 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:48:25.641274 (+ 644us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:48:25.641277 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:48:25.641670 (+ 393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:48:25.641671 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:48:25.641728 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:48:25.642196 (+ 468us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 13:48:25.642201 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 13:48:25.643915 (+ 1714us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.149.254:45043: Server oryx/127.13.149.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":69,"threads_started":1}
W20260924 13:48:25.644090 23207 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 13:48:25.639582 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.149.193:57613 (local address 127.13.149.254:45043)
0924 13:48:25.639688 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0924 13:48:25.639689 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0924 13:48:25.639697 (+ 8us) server_negotiation.cc:407] Connection header received
0924 13:48:25.639720 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:48:25.639721 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:48:25.639731 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:48:25.639796 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:48:25.640070 (+ 274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:48:25.640574 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:48:25.641334 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:48:25.641535 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:48:25.641588 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:48:25.644019 (+ 2431us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.149.193:57613: BlockingRecv error: recv got EOF from 127.13.149.193:57613 (error 108)
Metrics: {"server-negotiator.queue_time_us":88,"thread_start_us":52,"threads_started":1}
Sep 24 13:48:25 dist-test-slave-sjl2 krb5kdc[22164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.149.194@KRBTEST.COM for oryx/127.13.149.254@KRBTEST.COM, Server not found in Kerberos database
Sep 24 13:48:25 dist-test-slave-sjl2 krb5kdc[22164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.149.194@KRBTEST.COM for oryx/127.13.149.254@KRBTEST.COM, Server not found in Kerberos database
I20260924 13:48:25.781754 23208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:48:25.777223 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:45043 (local address 127.13.149.194:36821)
0924 13:48:25.777378 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0924 13:48:25.777389 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0924 13:48:25.777459 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:48:25.777587 (+ 128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:48:25.777588 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:48:25.777755 (+ 167us) client_negotiation.cc:189] Negotiated authn=SASL
0924 13:48:25.777804 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:48:25.777805 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:48:25.778487 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:48:25.778488 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:48:25.779030 (+ 542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:48:25.779032 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:48:25.779337 (+ 305us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:48:25.779338 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:48:25.779383 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:48:25.779795 (+ 412us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 13:48:25.779799 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0924 13:48:25.781679 (+ 1880us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.149.254:45043: Server oryx/127.13.149.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":73,"threads_started":1}
W20260924 13:48:25.781862 23207 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 13:48:25.777327 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.149.194:36821 (local address 127.13.149.254:45043)
0924 13:48:25.777398 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0924 13:48:25.777400 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:48:25.777449 (+ 49us) server_negotiation.cc:407] Connection header received
0924 13:48:25.777510 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:48:25.777511 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:48:25.777527 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:48:25.777575 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:48:25.777856 (+ 281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:48:25.778354 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:48:25.779098 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:48:25.779293 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:48:25.779337 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:48:25.781800 (+ 2463us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.149.194:36821: BlockingRecv error: recv got EOF from 127.13.149.194:36821 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
Sep 24 13:48:25 dist-test-slave-sjl2 krb5kdc[22164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.149.195@KRBTEST.COM for oryx/127.13.149.254@KRBTEST.COM, Server not found in Kerberos database
Sep 24 13:48:25 dist-test-slave-sjl2 krb5kdc[22164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.149.195@KRBTEST.COM for oryx/127.13.149.254@KRBTEST.COM, Server not found in Kerberos database
I20260924 13:48:25.904403 23209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:48:25.899761 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:45043 (local address 127.13.149.195:56467)
0924 13:48:25.899959 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0924 13:48:25.899968 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0924 13:48:25.900035 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:48:25.900168 (+ 133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:48:25.900169 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:48:25.900344 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0924 13:48:25.900388 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:48:25.900389 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:48:25.901107 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:48:25.901108 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:48:25.901673 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:48:25.901676 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:48:25.902014 (+ 338us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:48:25.902015 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:48:25.902071 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:48:25.902545 (+ 474us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 13:48:25.902549 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0924 13:48:25.904330 (+ 1781us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.149.254:45043: Server oryx/127.13.149.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":92,"threads_started":1}
W20260924 13:48:25.904510 23207 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 13:48:25.899835 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.149.195:56467 (local address 127.13.149.254:45043)
0924 13:48:25.899917 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0924 13:48:25.899919 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:48:25.900032 (+ 113us) server_negotiation.cc:407] Connection header received
0924 13:48:25.900092 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:48:25.900093 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:48:25.900108 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:48:25.900152 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:48:25.900458 (+ 306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:48:25.901033 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:48:25.901751 (+ 718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:48:25.901948 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:48:25.902001 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:48:25.904443 (+ 2442us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.149.195:56467: BlockingRecv error: recv got EOF from 127.13.149.195:56467 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
Sep 24 13:48:26 dist-test-slave-sjl2 krb5kdc[22164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.149.193@KRBTEST.COM for oryx/127.13.149.254@KRBTEST.COM, Server not found in Kerberos database
Sep 24 13:48:26 dist-test-slave-sjl2 krb5kdc[22164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.149.193@KRBTEST.COM for oryx/127.13.149.254@KRBTEST.COM, Server not found in Kerberos database
I20260924 13:48:26.649102 23210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:48:26.644559 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:45043 (local address 127.13.149.193:46223)
0924 13:48:26.644687 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0924 13:48:26.644697 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 13:48:26.644746 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:48:26.644884 (+ 138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:48:26.644885 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:48:26.645065 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0924 13:48:26.645114 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:48:26.645116 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:48:26.645717 (+ 601us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:48:26.645717 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:48:26.646230 (+ 513us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:48:26.646232 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:48:26.646560 (+ 328us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:48:26.646562 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:48:26.646615 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:48:26.647185 (+ 570us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 13:48:26.647190 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 13:48:26.649036 (+ 1846us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.149.254:45043: Server oryx/127.13.149.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":65,"threads_started":1}
W20260924 13:48:26.649235 23211 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 13:48:26.644630 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.149.193:46223 (local address 127.13.149.254:45043)
0924 13:48:26.644757 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0924 13:48:26.644759 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:48:26.644790 (+ 31us) server_negotiation.cc:407] Connection header received
0924 13:48:26.644813 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:48:26.644813 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:48:26.644824 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:48:26.644874 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:48:26.645169 (+ 295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:48:26.645661 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:48:26.646287 (+ 626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:48:26.646475 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:48:26.646521 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:48:26.649162 (+ 2641us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.149.193:46223: BlockingRecv error: recv got EOF from 127.13.149.193:46223 (error 108)
Metrics: {"server-negotiator.queue_time_us":110,"thread_start_us":58,"threads_started":1}
Sep 24 13:48:26 dist-test-slave-sjl2 krb5kdc[22164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.149.194@KRBTEST.COM for oryx/127.13.149.254@KRBTEST.COM, Server not found in Kerberos database
Sep 24 13:48:26 dist-test-slave-sjl2 krb5kdc[22164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.149.194@KRBTEST.COM for oryx/127.13.149.254@KRBTEST.COM, Server not found in Kerberos database
I20260924 13:48:26.786545 23212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:48:26.782173 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:45043 (local address 127.13.149.194:37407)
0924 13:48:26.782296 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0924 13:48:26.782308 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0924 13:48:26.782413 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:48:26.782538 (+ 125us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:48:26.782539 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:48:26.782722 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0924 13:48:26.782767 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:48:26.782769 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:48:26.783425 (+ 656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:48:26.783426 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:48:26.783939 (+ 513us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:48:26.783941 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:48:26.784274 (+ 333us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:48:26.784275 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:48:26.784334 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:48:26.784757 (+ 423us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 13:48:26.784762 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 13:48:26.786485 (+ 1723us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.149.254:45043: Server oryx/127.13.149.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":61,"threads_started":1}
W20260924 13:48:26.786645 23211 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 13:48:26.782237 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.149.194:37407 (local address 127.13.149.254:45043)
0924 13:48:26.782300 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0924 13:48:26.782302 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:48:26.782406 (+ 104us) server_negotiation.cc:407] Connection header received
0924 13:48:26.782467 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:48:26.782467 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:48:26.782480 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:48:26.782524 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:48:26.782840 (+ 316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:48:26.783352 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:48:26.784024 (+ 672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:48:26.784217 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:48:26.784272 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:48:26.786581 (+ 2309us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.149.194:37407: BlockingRecv error: recv got EOF from 127.13.149.194:37407 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
Sep 24 13:48:26 dist-test-slave-sjl2 krb5kdc[22164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.149.195@KRBTEST.COM for oryx/127.13.149.254@KRBTEST.COM, Server not found in Kerberos database
Sep 24 13:48:26 dist-test-slave-sjl2 krb5kdc[22164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.149.195@KRBTEST.COM for oryx/127.13.149.254@KRBTEST.COM, Server not found in Kerberos database
I20260924 13:48:26.909341 23213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:48:26.904887 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:45043 (local address 127.13.149.195:56973)
0924 13:48:26.905041 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0924 13:48:26.905055 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0924 13:48:26.905141 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:48:26.905263 (+ 122us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:48:26.905264 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:48:26.905456 (+ 192us) client_negotiation.cc:189] Negotiated authn=SASL
0924 13:48:26.905501 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:48:26.905503 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:48:26.906189 (+ 686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:48:26.906190 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:48:26.906730 (+ 540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:48:26.906732 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:48:26.907055 (+ 323us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:48:26.907057 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:48:26.907106 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:48:26.907545 (+ 439us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 13:48:26.907549 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0924 13:48:26.909281 (+ 1732us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.149.254:45043: Server oryx/127.13.149.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":86,"threads_started":1}
W20260924 13:48:26.909487 23211 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 13:48:26.905020 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.149.195:56973 (local address 127.13.149.254:45043)
0924 13:48:26.905082 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0924 13:48:26.905084 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:48:26.905124 (+ 40us) server_negotiation.cc:407] Connection header received
0924 13:48:26.905189 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:48:26.905190 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:48:26.905200 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:48:26.905246 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:48:26.905595 (+ 349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:48:26.906109 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:48:26.906798 (+ 689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:48:26.906980 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:48:26.907030 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:48:26.909412 (+ 2382us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.149.195:56973: BlockingRecv error: recv got EOF from 127.13.149.195:56973 (error 108)
Metrics: {"server-negotiator.queue_time_us":35}
Sep 24 13:48:27 dist-test-slave-sjl2 krb5kdc[22164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.149.193@KRBTEST.COM for oryx/127.13.149.254@KRBTEST.COM, Server not found in Kerberos database
Sep 24 13:48:27 dist-test-slave-sjl2 krb5kdc[22164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.149.193@KRBTEST.COM for oryx/127.13.149.254@KRBTEST.COM, Server not found in Kerberos database
I20260924 13:48:27.653790 23214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:48:27.649562 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:45043 (local address 127.13.149.193:40309)
0924 13:48:27.649693 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0924 13:48:27.649704 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0924 13:48:27.649755 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:48:27.649882 (+ 127us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:48:27.649882 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:48:27.650063 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0924 13:48:27.650112 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:48:27.650113 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:48:27.650730 (+ 617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:48:27.650730 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:48:27.651249 (+ 519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:48:27.651250 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:48:27.651556 (+ 306us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:48:27.651557 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:48:27.651594 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:48:27.652028 (+ 434us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 13:48:27.652032 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0924 13:48:27.653731 (+ 1699us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.149.254:45043: Server oryx/127.13.149.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":67,"threads_started":1}
W20260924 13:48:27.653957 23215 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 13:48:27.649647 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.149.193:40309 (local address 127.13.149.254:45043)
0924 13:48:27.649754 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0924 13:48:27.649755 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0924 13:48:27.649786 (+ 31us) server_negotiation.cc:407] Connection header received
0924 13:48:27.649807 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:48:27.649808 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:48:27.649819 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:48:27.649876 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:48:27.650167 (+ 291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:48:27.650672 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:48:27.651309 (+ 637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:48:27.651505 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:48:27.651554 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:48:27.653869 (+ 2315us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.149.193:40309: BlockingRecv error: recv got EOF from 127.13.149.193:40309 (error 108)
Metrics: {"server-negotiator.queue_time_us":83,"thread_start_us":44,"threads_started":1}
Sep 24 13:48:27 dist-test-slave-sjl2 krb5kdc[22164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.149.194@KRBTEST.COM for oryx/127.13.149.254@KRBTEST.COM, Server not found in Kerberos database
Sep 24 13:48:27 dist-test-slave-sjl2 krb5kdc[22164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.149.194@KRBTEST.COM for oryx/127.13.149.254@KRBTEST.COM, Server not found in Kerberos database
I20260924 13:48:27.791659 23216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:48:27.787071 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:45043 (local address 127.13.149.194:55027)
0924 13:48:27.787210 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0924 13:48:27.787223 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0924 13:48:27.787284 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:48:27.787514 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:48:27.787516 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:48:27.787679 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0924 13:48:27.787727 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:48:27.787728 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:48:27.788382 (+ 654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:48:27.788383 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:48:27.789022 (+ 639us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:48:27.789024 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:48:27.789464 (+ 440us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:48:27.789465 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:48:27.789515 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:48:27.789964 (+ 449us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 13:48:27.789968 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0924 13:48:27.791589 (+ 1621us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.149.254:45043: Server oryx/127.13.149.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":72,"threads_started":1}
W20260924 13:48:27.791756 23215 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 13:48:27.787247 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.149.194:55027 (local address 127.13.149.254:45043)
0924 13:48:27.787346 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0924 13:48:27.787347 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0924 13:48:27.787356 (+ 9us) server_negotiation.cc:407] Connection header received
0924 13:48:27.787386 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:48:27.787387 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:48:27.787398 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:48:27.787472 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:48:27.787810 (+ 338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:48:27.788300 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:48:27.789109 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:48:27.789393 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:48:27.789461 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:48:27.791691 (+ 2230us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.149.194:55027: BlockingRecv error: recv got EOF from 127.13.149.194:55027 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
Sep 24 13:48:27 dist-test-slave-sjl2 krb5kdc[22164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.149.195@KRBTEST.COM for oryx/127.13.149.254@KRBTEST.COM, Server not found in Kerberos database
Sep 24 13:48:27 dist-test-slave-sjl2 krb5kdc[22164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.149.195@KRBTEST.COM for oryx/127.13.149.254@KRBTEST.COM, Server not found in Kerberos database
I20260924 13:48:27.914136 23217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:48:27.909785 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:45043 (local address 127.13.149.195:36613)
0924 13:48:27.909911 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0924 13:48:27.909921 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 13:48:27.909977 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:48:27.910085 (+ 108us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:48:27.910086 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:48:27.910245 (+ 159us) client_negotiation.cc:189] Negotiated authn=SASL
0924 13:48:27.910291 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:48:27.910292 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:48:27.910907 (+ 615us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:48:27.910908 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:48:27.911433 (+ 525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:48:27.911435 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:48:27.911776 (+ 341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:48:27.911776 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:48:27.911812 (+ 36us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:48:27.912265 (+ 453us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 13:48:27.912269 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0924 13:48:27.914055 (+ 1786us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.149.254:45043: Server oryx/127.13.149.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":63,"threads_started":1}
W20260924 13:48:27.914278 23215 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 13:48:27.909856 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.149.195:36613 (local address 127.13.149.254:45043)
0924 13:48:27.909917 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0924 13:48:27.909919 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:48:27.909965 (+ 46us) server_negotiation.cc:407] Connection header received
0924 13:48:27.910016 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:48:27.910017 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:48:27.910029 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:48:27.910076 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:48:27.910335 (+ 259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:48:27.910841 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:48:27.911519 (+ 678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:48:27.911712 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:48:27.911768 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:48:27.914210 (+ 2442us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.149.195:36613: BlockingRecv error: recv got EOF from 127.13.149.195:36613 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
Sep 24 13:48:28 dist-test-slave-sjl2 krb5kdc[22164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.149.193@KRBTEST.COM for oryx/127.13.149.254@KRBTEST.COM, Server not found in Kerberos database
Sep 24 13:48:28 dist-test-slave-sjl2 krb5kdc[22164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.149.193@KRBTEST.COM for oryx/127.13.149.254@KRBTEST.COM, Server not found in Kerberos database
I20260924 13:48:28.658479 23218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:48:28.654280 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:45043 (local address 127.13.149.193:42773)
0924 13:48:28.654429 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0924 13:48:28.654439 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 13:48:28.654487 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:48:28.654619 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:48:28.654620 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:48:28.654782 (+ 162us) client_negotiation.cc:189] Negotiated authn=SASL
0924 13:48:28.654839 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:48:28.654841 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:48:28.655450 (+ 609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:48:28.655451 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:48:28.655996 (+ 545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:48:28.655998 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:48:28.656293 (+ 295us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:48:28.656293 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:48:28.656332 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:48:28.656755 (+ 423us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 13:48:28.656760 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 13:48:28.658403 (+ 1643us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.149.254:45043: Server oryx/127.13.149.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":69,"threads_started":1}
W20260924 13:48:28.658591 23219 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 13:48:28.654357 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.149.193:42773 (local address 127.13.149.254:45043)
0924 13:48:28.654464 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0924 13:48:28.654472 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0924 13:48:28.654481 (+ 9us) server_negotiation.cc:407] Connection header received
0924 13:48:28.654534 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:48:28.654534 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:48:28.654545 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:48:28.654608 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:48:28.654890 (+ 282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:48:28.655374 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:48:28.656057 (+ 683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:48:28.656243 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:48:28.656289 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:48:28.658509 (+ 2220us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.149.193:42773: BlockingRecv error: recv got EOF from 127.13.149.193:42773 (error 108)
Metrics: {"server-negotiator.queue_time_us":89,"thread_start_us":54,"threads_started":1}
Sep 24 13:48:28 dist-test-slave-sjl2 krb5kdc[22164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.149.194@KRBTEST.COM for oryx/127.13.149.254@KRBTEST.COM, Server not found in Kerberos database
Sep 24 13:48:28 dist-test-slave-sjl2 krb5kdc[22164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.149.194@KRBTEST.COM for oryx/127.13.149.254@KRBTEST.COM, Server not found in Kerberos database
I20260924 13:48:28.800295 23220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:48:28.792121 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:45043 (local address 127.13.149.194:52767)
0924 13:48:28.792267 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0924 13:48:28.792278 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0924 13:48:28.792362 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:48:28.792491 (+ 129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:48:28.792492 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:48:28.792669 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0924 13:48:28.792713 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:48:28.792715 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:48:28.793385 (+ 670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:48:28.793386 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:48:28.793921 (+ 535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:48:28.793922 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:48:28.794264 (+ 342us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:48:28.794264 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:48:28.794318 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:48:28.798236 (+ 3918us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 13:48:28.798242 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0924 13:48:28.800215 (+ 1973us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.149.254:45043: Server oryx/127.13.149.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":62,"threads_started":1}
W20260924 13:48:28.800422 23219 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 13:48:28.792194 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.149.194:52767 (local address 127.13.149.254:45043)
0924 13:48:28.792238 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0924 13:48:28.792240 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:48:28.792342 (+ 102us) server_negotiation.cc:407] Connection header received
0924 13:48:28.792419 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:48:28.792419 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:48:28.792433 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:48:28.792481 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:48:28.792787 (+ 306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:48:28.793317 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:48:28.793996 (+ 679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:48:28.794203 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:48:28.794259 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:48:28.800367 (+ 6108us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.149.194:52767: BlockingRecv error: recv got EOF from 127.13.149.194:52767 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
Sep 24 13:48:28 dist-test-slave-sjl2 krb5kdc[22164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.149.195@KRBTEST.COM for oryx/127.13.149.254@KRBTEST.COM, Server not found in Kerberos database
Sep 24 13:48:28 dist-test-slave-sjl2 krb5kdc[22164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.149.195@KRBTEST.COM for oryx/127.13.149.254@KRBTEST.COM, Server not found in Kerberos database
I20260924 13:48:28.919164 23221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:48:28.914568 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:45043 (local address 127.13.149.195:43247)
0924 13:48:28.914707 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0924 13:48:28.914716 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0924 13:48:28.914772 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:48:28.914898 (+ 126us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:48:28.914898 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:48:28.915059 (+ 161us) client_negotiation.cc:189] Negotiated authn=SASL
0924 13:48:28.915105 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:48:28.915107 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:48:28.915740 (+ 633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:48:28.915742 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:48:28.916356 (+ 614us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:48:28.916359 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:48:28.916735 (+ 376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:48:28.916737 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:48:28.916788 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:48:28.917267 (+ 479us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 13:48:28.917271 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0924 13:48:28.919100 (+ 1829us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.149.254:45043: Server oryx/127.13.149.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":66,"threads_started":1}
W20260924 13:48:28.919274 23219 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 13:48:28.914627 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.149.195:43247 (local address 127.13.149.254:45043)
0924 13:48:28.914676 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0924 13:48:28.914678 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:48:28.914767 (+ 89us) server_negotiation.cc:407] Connection header received
0924 13:48:28.914826 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:48:28.914827 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:48:28.914838 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:48:28.914875 (+ 37us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:48:28.915155 (+ 280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:48:28.915649 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:48:28.916450 (+ 801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:48:28.916652 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:48:28.916705 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:48:28.919202 (+ 2497us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.149.195:43247: BlockingRecv error: recv got EOF from 127.13.149.195:43247 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
Sep 24 13:48:29 dist-test-slave-sjl2 krb5kdc[22164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.149.193@KRBTEST.COM for oryx/127.13.149.254@KRBTEST.COM, Server not found in Kerberos database
Sep 24 13:48:29 dist-test-slave-sjl2 krb5kdc[22164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.149.193@KRBTEST.COM for oryx/127.13.149.254@KRBTEST.COM, Server not found in Kerberos database
I20260924 13:48:29.663908 23222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:48:29.659081 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:45043 (local address 127.13.149.193:39667)
0924 13:48:29.659199 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0924 13:48:29.659209 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 13:48:29.659281 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:48:29.659432 (+ 151us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:48:29.659433 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:48:29.659593 (+ 160us) client_negotiation.cc:189] Negotiated authn=SASL
0924 13:48:29.659644 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:48:29.659646 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:48:29.660369 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:48:29.660371 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:48:29.660908 (+ 537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:48:29.660910 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:48:29.661333 (+ 423us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:48:29.661334 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:48:29.661394 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:48:29.661890 (+ 496us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 13:48:29.661894 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0924 13:48:29.663832 (+ 1938us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.149.254:45043: Server oryx/127.13.149.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":69,"threads_started":1}
W20260924 13:48:29.664009 23223 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 13:48:29.659193 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.149.193:39667 (local address 127.13.149.254:45043)
0924 13:48:29.659311 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0924 13:48:29.659313 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:48:29.659322 (+ 9us) server_negotiation.cc:407] Connection header received
0924 13:48:29.659348 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:48:29.659348 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:48:29.659359 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:48:29.659418 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:48:29.659746 (+ 328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:48:29.660277 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:48:29.661055 (+ 778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:48:29.661266 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:48:29.661323 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:48:29.663935 (+ 2612us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.149.193:39667: BlockingRecv error: recv got EOF from 127.13.149.193:39667 (error 108)
Metrics: {"server-negotiator.queue_time_us":92,"thread_start_us":49,"threads_started":1}
Sep 24 13:48:29 dist-test-slave-sjl2 krb5kdc[22164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.149.194@KRBTEST.COM for oryx/127.13.149.254@KRBTEST.COM, Server not found in Kerberos database
Sep 24 13:48:29 dist-test-slave-sjl2 krb5kdc[22164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.149.194@KRBTEST.COM for oryx/127.13.149.254@KRBTEST.COM, Server not found in Kerberos database
I20260924 13:48:29.805135 23224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:48:29.800864 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:45043 (local address 127.13.149.194:49447)
0924 13:48:29.801040 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0924 13:48:29.801051 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0924 13:48:29.801113 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:48:29.801245 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:48:29.801246 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:48:29.801399 (+ 153us) client_negotiation.cc:189] Negotiated authn=SASL
0924 13:48:29.801448 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:48:29.801450 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:48:29.802095 (+ 645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:48:29.802095 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:48:29.802626 (+ 531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:48:29.802628 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:48:29.802920 (+ 292us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:48:29.802920 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:48:29.802957 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:48:29.803411 (+ 454us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 13:48:29.803416 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 13:48:29.805068 (+ 1652us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.149.254:45043: Server oryx/127.13.149.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":102,"threads_started":1}
W20260924 13:48:29.805217 23223 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 13:48:29.800970 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.149.194:49447 (local address 127.13.149.254:45043)
0924 13:48:29.801059 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0924 13:48:29.801061 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:48:29.801109 (+ 48us) server_negotiation.cc:407] Connection header received
0924 13:48:29.801160 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:48:29.801161 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:48:29.801171 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:48:29.801231 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:48:29.801514 (+ 283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:48:29.802031 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:48:29.802685 (+ 654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:48:29.802873 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:48:29.802917 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:48:29.805163 (+ 2246us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.149.194:49447: BlockingRecv error: recv got EOF from 127.13.149.194:49447 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
I20260924 13:48:29.849295 13911 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJN9WRF/build/release/bin/kudu with pid 22747
I20260924 13:48:29.853780 13911 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJN9WRF/build/release/bin/kudu with pid 22899
I20260924 13:48:29.857663 13911 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJN9WRF/build/release/bin/kudu with pid 23031
I20260924 13:48:29.861429 13911 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJN9WRF/build/release/bin/kudu with pid 22677
2026-09-24T13:48:29Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (17381 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-09-24T13:48:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T13:48:29Z Disabled control of system clock
I20260924 13:48:29.887327 13911 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJN9WRF/build/release/bin/kudu
/tmp/dist-test-taskJN9WRF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790257629041645-13911-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790257629041645-13911-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790257629041645-13911-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790257629041645-13911-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.149.254:44415
--webserver_interface=127.13.149.254
--webserver_port=0
--builtin_ntp_servers=127.13.149.212:39327
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.149.254:44415
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260924 13:48:29.967650 23228 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 13:48:29.967852 23228 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 13:48:29.967871 23228 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 13:48:29.969106 23228 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 13:48:29.969149 23228 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 13:48:29.969161 23228 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 13:48:29.969172 23228 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 13:48:29.969182 23228 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 13:48:29.970530 23228 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.149.212:39327
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790257629041645-13911-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790257629041645-13911-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.149.254:44415
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.149.254:44415
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790257629041645-13911-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.149.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23228
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790257629041645-13911-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 12:29:37 UTC on ccycloud.quasar-iqzbxn.root.comops.site
build id 4670106
I20260924 13:48:29.970764 23228 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 13:48:29.971050 23228 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 13:48:29.976315 23236 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 13:48:29.976363 23233 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 13:48:29.976351 23234 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 13:48:29.976377 23228 server_base.cc:1034] running on GCE node
I20260924 13:48:29.976773 23228 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 13:48:29.977097 23228 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 13:48:29.978235 23228 hybrid_clock.cc:648] HybridClock initialized: now 1790257709978222 us; error 33 us; skew 500 ppm
I20260924 13:48:29.979430 23228 webserver.cc:460] Webserver started at http://127.13.149.254:46683/ using document root <none> and password file <none>
I20260924 13:48:29.979653 23228 fs_manager.cc:359] Metadata directory not provided
I20260924 13:48:29.979702 23228 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 13:48:29.979825 23228 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 13:48:29.980944 23228 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790257629041645-13911-0/minicluster-data/master-0/data/instance:
uuid: "0d3f2b53c4ad4ea28883a2e3adf0f5bb"
format_stamp: "Formatted at 2026-09-24 13:48:29 on dist-test-slave-sjl2"
server_key: "75dfe23350bc80bbe2375bd7df10760b"
server_key_iv: "4729dfa8b7fe679d04bb40b0bd4ecab3"
server_key_version: "encryptionkey@0"
I20260924 13:48:29.981305 23228 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790257629041645-13911-0/minicluster-data/master-0/wal/instance:
uuid: "0d3f2b53c4ad4ea28883a2e3adf0f5bb"
format_stamp: "Formatted at 2026-09-24 13:48:29 on dist-test-slave-sjl2"
server_key: "75dfe23350bc80bbe2375bd7df10760b"
server_key_iv: "4729dfa8b7fe679d04bb40b0bd4ecab3"
server_key_version: "encryptionkey@0"
I20260924 13:48:29.982542 23228 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260924 13:48:29.983289 23242 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 13:48:29.983443 23228 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 13:48:29.983527 23228 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790257629041645-13911-0/minicluster-data/master-0/data,/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790257629041645-13911-0/minicluster-data/master-0/wal
uuid: "0d3f2b53c4ad4ea28883a2e3adf0f5bb"
format_stamp: "Formatted at 2026-09-24 13:48:29 on dist-test-slave-sjl2"
server_key: "75dfe23350bc80bbe2375bd7df10760b"
server_key_iv: "4729dfa8b7fe679d04bb40b0bd4ecab3"
server_key_version: "encryptionkey@0"
I20260924 13:48:29.983590 23228 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790257629041645-13911-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790257629041645-13911-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790257629041645-13911-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 13:48:29.994859 23228 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 13:48:29.995184 23228 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 13:48:29.995314 23228 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 13:48:29.999553 23228 rpc_server.cc:307] RPC server started. Bound to: 127.13.149.254:44415
I20260924 13:48:29.999652 23294 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.149.254:44415 every 8 connection(s)
I20260924 13:48:29.999998 23228 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790257629041645-13911-0/minicluster-data/master-0/data/info.pb
I20260924 13:48:30.001216 23295 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 13:48:30.001525 13911 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJN9WRF/build/release/bin/kudu as pid 23228
I20260924 13:48:30.001622 13911 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790257629041645-13911-0/minicluster-data/master-0/wal/instance
I20260924 13:48:30.001809 13911 external_mini_cluster.cc:1428] Setting key 5ff5c8197a96aa91c81d71fdf53a5c21
I20260924 13:48:30.004508 23295 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0d3f2b53c4ad4ea28883a2e3adf0f5bb: Bootstrap starting.
I20260924 13:48:30.005179 23295 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0d3f2b53c4ad4ea28883a2e3adf0f5bb: Neither blocks nor log segments found. Creating new log.
I20260924 13:48:30.005434 23295 log.cc:824] T 00000000000000000000000000000000 P 0d3f2b53c4ad4ea28883a2e3adf0f5bb: Log is configured to *not* fsync() on all Append() calls
I20260924 13:48:30.006104 23295 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0d3f2b53c4ad4ea28883a2e3adf0f5bb: No bootstrap required, opened a new log
I20260924 13:48:30.007462 23295 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0d3f2b53c4ad4ea28883a2e3adf0f5bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d3f2b53c4ad4ea28883a2e3adf0f5bb" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 44415 } }
I20260924 13:48:30.007606 23295 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0d3f2b53c4ad4ea28883a2e3adf0f5bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 13:48:30.007637 23295 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0d3f2b53c4ad4ea28883a2e3adf0f5bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d3f2b53c4ad4ea28883a2e3adf0f5bb, State: Initialized, Role: FOLLOWER
I20260924 13:48:30.007932 23298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:48:30.002714 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42514 (local address 127.13.149.254:44415)
0924 13:48:30.002990 (+ 276us) server_negotiation.cc:206] Beginning negotiation
0924 13:48:30.003001 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0924 13:48:30.003047 (+ 46us) server_negotiation.cc:407] Connection header received
0924 13:48:30.003671 (+ 624us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:48:30.003681 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:48:30.003756 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:48:30.003899 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:48:30.004229 (+ 330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:48:30.005214 (+ 985us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:48:30.006058 (+ 844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:48:30.006502 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:48:30.006614 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:48:30.007261 (+ 647us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:48:30.007265 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:48:30.007265 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 13:48:30.007429 (+ 164us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:48:30.007464 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 13:48:30.007475 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:48:30.007531 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0924 13:48:30.007608 (+ 77us) server_negotiation.cc:299] Negotiation successful
0924 13:48:30.007691 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":180,"thread_start_us":119,"threads_started":1}
I20260924 13:48:30.007787 23295 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0d3f2b53c4ad4ea28883a2e3adf0f5bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d3f2b53c4ad4ea28883a2e3adf0f5bb" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 44415 } }
I20260924 13:48:30.008100 23295 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0d3f2b53c4ad4ea28883a2e3adf0f5bb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 13:48:30.008144 23295 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0d3f2b53c4ad4ea28883a2e3adf0f5bb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 13:48:30.008190 23295 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0d3f2b53c4ad4ea28883a2e3adf0f5bb [term 0 FOLLOWER]: Advancing to term 1
I20260924 13:48:30.008785 23295 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0d3f2b53c4ad4ea28883a2e3adf0f5bb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d3f2b53c4ad4ea28883a2e3adf0f5bb" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 44415 } }
I20260924 13:48:30.008888 23295 leader_election.cc:302] T 00000000000000000000000000000000 P 0d3f2b53c4ad4ea28883a2e3adf0f5bb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0d3f2b53c4ad4ea28883a2e3adf0f5bb; no voters:
I20260924 13:48:30.009074 23295 leader_election.cc:288] T 00000000000000000000000000000000 P 0d3f2b53c4ad4ea28883a2e3adf0f5bb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 13:48:30.009171 23300 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0d3f2b53c4ad4ea28883a2e3adf0f5bb [term 1 FOLLOWER]: Leader election won for term 1
I20260924 13:48:30.009336 23295 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0d3f2b53c4ad4ea28883a2e3adf0f5bb [sys.catalog]: configured and running, proceeding with master startup.
I20260924 13:48:30.009377 23300 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0d3f2b53c4ad4ea28883a2e3adf0f5bb [term 1 LEADER]: Becoming Leader. State: Replica: 0d3f2b53c4ad4ea28883a2e3adf0f5bb, State: Running, Role: LEADER
I20260924 13:48:30.009470 23300 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0d3f2b53c4ad4ea28883a2e3adf0f5bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d3f2b53c4ad4ea28883a2e3adf0f5bb" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 44415 } }
I20260924 13:48:30.009850 23301 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0d3f2b53c4ad4ea28883a2e3adf0f5bb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0d3f2b53c4ad4ea28883a2e3adf0f5bb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d3f2b53c4ad4ea28883a2e3adf0f5bb" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 44415 } } }
I20260924 13:48:30.009948 23301 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0d3f2b53c4ad4ea28883a2e3adf0f5bb [sys.catalog]: This master's current role is: LEADER
I20260924 13:48:30.010149 23309 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 13:48:30.010169 23302 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0d3f2b53c4ad4ea28883a2e3adf0f5bb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0d3f2b53c4ad4ea28883a2e3adf0f5bb. Latest consensus state: current_term: 1 leader_uuid: "0d3f2b53c4ad4ea28883a2e3adf0f5bb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d3f2b53c4ad4ea28883a2e3adf0f5bb" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 44415 } } }
I20260924 13:48:30.010252 23302 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0d3f2b53c4ad4ea28883a2e3adf0f5bb [sys.catalog]: This master's current role is: LEADER
I20260924 13:48:30.010764 23309 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 13:48:30.012619 23309 catalog_manager.cc:1347] Generated new cluster ID: 2128c200b3ac4f79a2210fd454515b47
I20260924 13:48:30.012676 23309 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 13:48:30.028111 23309 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 13:48:30.028849 23309 catalog_manager.cc:1504] Loading token signing keys...
I20260924 13:48:30.033644 23309 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0d3f2b53c4ad4ea28883a2e3adf0f5bb: Generated new TSK 0
I20260924 13:48:30.033838 23309 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260924 13:48:30.036849 13911 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJN9WRF/build/release/bin/kudu
/tmp/dist-test-taskJN9WRF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790257629041645-13911-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790257629041645-13911-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790257629041645-13911-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790257629041645-13911-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.149.193:0
--local_ip_for_outbound_sockets=127.13.149.193
--webserver_interface=127.13.149.193
--webserver_port=0
--tserver_master_addrs=127.13.149.254:44415
--builtin_ntp_servers=127.13.149.212:39327
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260924 13:48:30.119639 23319 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 13:48:30.119866 23319 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 13:48:30.119889 23319 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 13:48:30.121167 23319 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 13:48:30.121208 23319 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 13:48:30.121235 23319 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.149.193
I20260924 13:48:30.122624 23319 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.149.212:39327
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790257629041645-13911-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790257629041645-13911-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.149.193:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790257629041645-13911-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.149.193
--webserver_port=0
--tserver_master_addrs=127.13.149.254:44415
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23319
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.149.193
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790257629041645-13911-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 12:29:37 UTC on ccycloud.quasar-iqzbxn.root.comops.site
build id 4670106
I20260924 13:48:30.122869 23319 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 13:48:30.123183 23319 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 13:48:30.128074 23325 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 13:48:30.128074 23324 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 13:48:30.128114 23327 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 13:48:30.128268 23319 server_base.cc:1034] running on GCE node
I20260924 13:48:30.128563 23319 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 13:48:30.128798 23319 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 13:48:30.129937 23319 hybrid_clock.cc:648] HybridClock initialized: now 1790257710129916 us; error 38 us; skew 500 ppm
I20260924 13:48:30.131167 23319 webserver.cc:460] Webserver started at http://127.13.149.193:36801/ using document root <none> and password file <none>
I20260924 13:48:30.131410 23319 fs_manager.cc:359] Metadata directory not provided
I20260924 13:48:30.131487 23319 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 13:48:30.131620 23319 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 13:48:30.132691 23319 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790257629041645-13911-0/minicluster-data/ts-0/data/instance:
uuid: "cacf7e7cc3624a28a3bc36302f1578a6"
format_stamp: "Formatted at 2026-09-24 13:48:30 on dist-test-slave-sjl2"
server_key: "03c81f9df7c4ea70c10afdfedcab40ce"
server_key_iv: "6127524b18fc9ed2329374127b3e55a4"
server_key_version: "encryptionkey@0"
I20260924 13:48:30.133056 23319 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790257629041645-13911-0/minicluster-data/ts-0/wal/instance:
uuid: "cacf7e7cc3624a28a3bc36302f1578a6"
format_stamp: "Formatted at 2026-09-24 13:48:30 on dist-test-slave-sjl2"
server_key: "03c81f9df7c4ea70c10afdfedcab40ce"
server_key_iv: "6127524b18fc9ed2329374127b3e55a4"
server_key_version: "encryptionkey@0"
I20260924 13:48:30.134348 23319 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260924 13:48:30.135061 23333 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 13:48:30.135218 23319 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 13:48:30.135293 23319 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790257629041645-13911-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790257629041645-13911-0/minicluster-data/ts-0/wal
uuid: "cacf7e7cc3624a28a3bc36302f1578a6"
format_stamp: "Formatted at 2026-09-24 13:48:30 on dist-test-slave-sjl2"
server_key: "03c81f9df7c4ea70c10afdfedcab40ce"
server_key_iv: "6127524b18fc9ed2329374127b3e55a4"
server_key_version: "encryptionkey@0"
I20260924 13:48:30.135366 23319 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790257629041645-13911-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790257629041645-13911-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790257629041645-13911-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 13:48:30.155618 23319 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 13:48:30.155933 23319 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 13:48:30.156060 23319 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 13:48:30.156281 23319 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 13:48:30.156607 23319 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 13:48:30.156647 23319 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 13:48:30.156677 23319 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 13:48:30.156700 23319 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 13:48:30.162379 23319 rpc_server.cc:307] RPC server started. Bound to: 127.13.149.193:43209
I20260924 13:48:30.162424 23446 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.149.193:43209 every 8 connection(s)
I20260924 13:48:30.162747 23319 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790257629041645-13911-0/minicluster-data/ts-0/data/info.pb
I20260924 13:48:30.168047 23298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:48:30.163480 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.149.193:48147 (local address 127.13.149.254:44415)
0924 13:48:30.163539 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0924 13:48:30.163541 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:48:30.163775 (+ 234us) server_negotiation.cc:407] Connection header received
0924 13:48:30.164340 (+ 565us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:48:30.164342 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:48:30.164353 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:48:30.164411 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:48:30.165227 (+ 816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:48:30.165756 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:48:30.166705 (+ 949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:48:30.166912 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:48:30.166968 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:48:30.167705 (+ 737us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:48:30.167706 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:48:30.167706 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 13:48:30.167711 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:48:30.167725 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 13:48:30.167729 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:48:30.167781 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0924 13:48:30.167949 (+ 168us) server_negotiation.cc:299] Negotiation successful
0924 13:48:30.167981 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260924 13:48:30.168222 23449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:48:30.163473 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:44415 (local address 127.13.149.193:48147)
0924 13:48:30.163641 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0924 13:48:30.163675 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0924 13:48:30.164212 (+ 537us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:48:30.164496 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:48:30.164500 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:48:30.164511 (+ 11us) client_negotiation.cc:189] Negotiated authn=SASL
0924 13:48:30.165128 (+ 617us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:48:30.165133 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:48:30.165844 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:48:30.165844 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:48:30.166626 (+ 782us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:48:30.166629 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:48:30.166975 (+ 346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:48:30.166976 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:48:30.167078 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:48:30.167558 (+ 480us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0924 13:48:30.167558 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0924 13:48:30.167588 (+ 30us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0924 13:48:30.167611 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 13:48:30.167820 (+ 209us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 13:48:30.167828 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 13:48:30.167831 (+ 3us) client_negotiation.cc:769] Sending connection context
0924 13:48:30.167906 (+ 75us) client_negotiation.cc:240] Negotiation successful
0924 13:48:30.167995 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":78,"threads_started":1}
I20260924 13:48:30.168745 23447 heartbeater.cc:347] Connected to a master server at 127.13.149.254:44415
I20260924 13:48:30.168834 23447 heartbeater.cc:464] Registering TS with master...
I20260924 13:48:30.169051 23447 heartbeater.cc:511] Master 127.13.149.254:44415 requested a full tablet report, sending...
I20260924 13:48:30.169639 23259 ts_manager.cc:194] Registered new tserver with Master: cacf7e7cc3624a28a3bc36302f1578a6 (127.13.149.193:43209)
I20260924 13:48:30.170222 23259 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.13.149.193:48147
I20260924 13:48:30.171707 13911 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJN9WRF/build/release/bin/kudu as pid 23319
I20260924 13:48:30.171782 13911 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790257629041645-13911-0/minicluster-data/ts-0/wal/instance
I20260924 13:48:30.171950 13911 external_mini_cluster.cc:1428] Setting key 29e235b7ddeec05aeb20d7d4f6816ae4
I20260924 13:48:30.173022 13911 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJN9WRF/build/release/bin/kudu
/tmp/dist-test-taskJN9WRF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790257629041645-13911-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790257629041645-13911-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790257629041645-13911-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790257629041645-13911-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.149.194:0
--local_ip_for_outbound_sockets=127.13.149.194
--webserver_interface=127.13.149.194
--webserver_port=0
--tserver_master_addrs=127.13.149.254:44415
--builtin_ntp_servers=127.13.149.212:39327
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260924 13:48:30.253736 23450 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 13:48:30.253958 23450 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 13:48:30.253980 23450 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 13:48:30.255236 23450 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 13:48:30.255290 23450 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 13:48:30.255316 23450 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.149.194
I20260924 13:48:30.256668 23450 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.149.212:39327
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790257629041645-13911-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790257629041645-13911-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.149.194:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790257629041645-13911-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.149.194
--webserver_port=0
--tserver_master_addrs=127.13.149.254:44415
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23450
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.149.194
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790257629041645-13911-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 12:29:37 UTC on ccycloud.quasar-iqzbxn.root.comops.site
build id 4670106
I20260924 13:48:30.256881 23450 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 13:48:30.257212 23450 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 13:48:30.262152 23456 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 13:48:30.262148 23455 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 13:48:30.262210 23450 server_base.cc:1034] running on GCE node
W20260924 13:48:30.262179 23458 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 13:48:30.262581 23450 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 13:48:30.262816 23450 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 13:48:30.263955 23450 hybrid_clock.cc:648] HybridClock initialized: now 1790257710263947 us; error 29 us; skew 500 ppm
I20260924 13:48:30.265257 23450 webserver.cc:460] Webserver started at http://127.13.149.194:41431/ using document root <none> and password file <none>
I20260924 13:48:30.265493 23450 fs_manager.cc:359] Metadata directory not provided
I20260924 13:48:30.265551 23450 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 13:48:30.265682 23450 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 13:48:30.266742 23450 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790257629041645-13911-0/minicluster-data/ts-1/data/instance:
uuid: "657c0099712743bf93a77da84bde6ca2"
format_stamp: "Formatted at 2026-09-24 13:48:30 on dist-test-slave-sjl2"
server_key: "6366b6d35fd0b37fbce7f2ba90b158d9"
server_key_iv: "c81882a494301237e476f4378bdec072"
server_key_version: "encryptionkey@0"
I20260924 13:48:30.267112 23450 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790257629041645-13911-0/minicluster-data/ts-1/wal/instance:
uuid: "657c0099712743bf93a77da84bde6ca2"
format_stamp: "Formatted at 2026-09-24 13:48:30 on dist-test-slave-sjl2"
server_key: "6366b6d35fd0b37fbce7f2ba90b158d9"
server_key_iv: "c81882a494301237e476f4378bdec072"
server_key_version: "encryptionkey@0"
I20260924 13:48:30.268321 23450 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260924 13:48:30.268978 23464 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 13:48:30.269119 23450 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260924 13:48:30.269198 23450 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790257629041645-13911-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790257629041645-13911-0/minicluster-data/ts-1/wal
uuid: "657c0099712743bf93a77da84bde6ca2"
format_stamp: "Formatted at 2026-09-24 13:48:30 on dist-test-slave-sjl2"
server_key: "6366b6d35fd0b37fbce7f2ba90b158d9"
server_key_iv: "c81882a494301237e476f4378bdec072"
server_key_version: "encryptionkey@0"
I20260924 13:48:30.269268 23450 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790257629041645-13911-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790257629041645-13911-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790257629041645-13911-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 13:48:30.286119 23450 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 13:48:30.286435 23450 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 13:48:30.286566 23450 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 13:48:30.286770 23450 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 13:48:30.287091 23450 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 13:48:30.287124 23450 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 13:48:30.287153 23450 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 13:48:30.287177 23450 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 13:48:30.292594 23450 rpc_server.cc:307] RPC server started. Bound to: 127.13.149.194:40249
I20260924 13:48:30.292641 23577 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.149.194:40249 every 8 connection(s)
I20260924 13:48:30.292984 23450 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790257629041645-13911-0/minicluster-data/ts-1/data/info.pb
I20260924 13:48:30.297657 13911 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJN9WRF/build/release/bin/kudu as pid 23450
I20260924 13:48:30.297755 13911 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790257629041645-13911-0/minicluster-data/ts-1/wal/instance
I20260924 13:48:30.297937 13911 external_mini_cluster.cc:1428] Setting key 494c9cf975fa995596cdd890ba9b72f3
I20260924 13:48:30.297945 23298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:48:30.293701 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.149.194:58709 (local address 127.13.149.254:44415)
0924 13:48:30.293768 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0924 13:48:30.293770 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:48:30.294006 (+ 236us) server_negotiation.cc:407] Connection header received
0924 13:48:30.294474 (+ 468us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:48:30.294475 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:48:30.294484 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:48:30.294559 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:48:30.295191 (+ 632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:48:30.295672 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:48:30.296596 (+ 924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:48:30.296768 (+ 172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:48:30.296815 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:48:30.297640 (+ 825us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:48:30.297642 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:48:30.297642 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 13:48:30.297647 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:48:30.297663 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 13:48:30.297667 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:48:30.297710 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0924 13:48:30.297835 (+ 125us) server_negotiation.cc:299] Negotiation successful
0924 13:48:30.297865 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260924 13:48:30.298123 23580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:48:30.293683 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:44415 (local address 127.13.149.194:58709)
0924 13:48:30.293889 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0924 13:48:30.293930 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0924 13:48:30.294402 (+ 472us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:48:30.294620 (+ 218us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:48:30.294624 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:48:30.294631 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0924 13:48:30.295130 (+ 499us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:48:30.295133 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:48:30.295734 (+ 601us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:48:30.295735 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:48:30.296531 (+ 796us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:48:30.296534 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:48:30.296856 (+ 322us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:48:30.296858 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:48:30.297017 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:48:30.297481 (+ 464us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0924 13:48:30.297481 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0924 13:48:30.297509 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0924 13:48:30.297533 (+ 24us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 13:48:30.297717 (+ 184us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 13:48:30.297722 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 13:48:30.297725 (+ 3us) client_negotiation.cc:769] Sending connection context
0924 13:48:30.297789 (+ 64us) client_negotiation.cc:240] Negotiation successful
0924 13:48:30.297894 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":84,"threads_started":1}
I20260924 13:48:30.298552 23578 heartbeater.cc:347] Connected to a master server at 127.13.149.254:44415
I20260924 13:48:30.298732 23578 heartbeater.cc:464] Registering TS with master...
I20260924 13:48:30.298846 13911 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJN9WRF/build/release/bin/kudu
/tmp/dist-test-taskJN9WRF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790257629041645-13911-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790257629041645-13911-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790257629041645-13911-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790257629041645-13911-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.149.195:0
--local_ip_for_outbound_sockets=127.13.149.195
--webserver_interface=127.13.149.195
--webserver_port=0
--tserver_master_addrs=127.13.149.254:44415
--builtin_ntp_servers=127.13.149.212:39327
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260924 13:48:30.298971 23578 heartbeater.cc:511] Master 127.13.149.254:44415 requested a full tablet report, sending...
I20260924 13:48:30.299394 23259 ts_manager.cc:194] Registered new tserver with Master: 657c0099712743bf93a77da84bde6ca2 (127.13.149.194:40249)
I20260924 13:48:30.299754 23259 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.13.149.194:58709
W20260924 13:48:30.377272 23581 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 13:48:30.377485 23581 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 13:48:30.377504 23581 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 13:48:30.378659 23581 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 13:48:30.378705 23581 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 13:48:30.378728 23581 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.149.195
I20260924 13:48:30.380064 23581 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.149.212:39327
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790257629041645-13911-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790257629041645-13911-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.149.195:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790257629041645-13911-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.149.195
--webserver_port=0
--tserver_master_addrs=127.13.149.254:44415
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23581
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.149.195
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790257629041645-13911-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 12:29:37 UTC on ccycloud.quasar-iqzbxn.root.comops.site
build id 4670106
I20260924 13:48:30.380288 23581 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 13:48:30.380563 23581 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260924 13:48:30.385520 23581 server_base.cc:1034] running on GCE node
W20260924 13:48:30.385478 23586 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 13:48:30.385478 23587 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 13:48:30.385485 23589 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 13:48:30.385903 23581 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 13:48:30.386137 23581 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 13:48:30.387276 23581 hybrid_clock.cc:648] HybridClock initialized: now 1790257710387259 us; error 35 us; skew 500 ppm
I20260924 13:48:30.388518 23581 webserver.cc:460] Webserver started at http://127.13.149.195:39399/ using document root <none> and password file <none>
I20260924 13:48:30.388733 23581 fs_manager.cc:359] Metadata directory not provided
I20260924 13:48:30.388779 23581 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 13:48:30.388885 23581 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 13:48:30.390013 23581 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790257629041645-13911-0/minicluster-data/ts-2/data/instance:
uuid: "f163833346484109b7324f1d7aa92e2c"
format_stamp: "Formatted at 2026-09-24 13:48:30 on dist-test-slave-sjl2"
server_key: "c10437c909b67ddd6f40711f0e370575"
server_key_iv: "92556b354bc9ee8c9c3c0bcc45cd789b"
server_key_version: "encryptionkey@0"
I20260924 13:48:30.390360 23581 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790257629041645-13911-0/minicluster-data/ts-2/wal/instance:
uuid: "f163833346484109b7324f1d7aa92e2c"
format_stamp: "Formatted at 2026-09-24 13:48:30 on dist-test-slave-sjl2"
server_key: "c10437c909b67ddd6f40711f0e370575"
server_key_iv: "92556b354bc9ee8c9c3c0bcc45cd789b"
server_key_version: "encryptionkey@0"
I20260924 13:48:30.391592 23581 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260924 13:48:30.392247 23595 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 13:48:30.392436 23581 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 13:48:30.392550 23581 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790257629041645-13911-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790257629041645-13911-0/minicluster-data/ts-2/wal
uuid: "f163833346484109b7324f1d7aa92e2c"
format_stamp: "Formatted at 2026-09-24 13:48:30 on dist-test-slave-sjl2"
server_key: "c10437c909b67ddd6f40711f0e370575"
server_key_iv: "92556b354bc9ee8c9c3c0bcc45cd789b"
server_key_version: "encryptionkey@0"
I20260924 13:48:30.392622 23581 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790257629041645-13911-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790257629041645-13911-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790257629041645-13911-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 13:48:30.407948 23581 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 13:48:30.408241 23581 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 13:48:30.408365 23581 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 13:48:30.408572 23581 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 13:48:30.408902 23581 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 13:48:30.409013 23581 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 13:48:30.409049 23581 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 13:48:30.409072 23581 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 13:48:30.414395 23581 rpc_server.cc:307] RPC server started. Bound to: 127.13.149.195:46061
I20260924 13:48:30.414454 23708 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.149.195:46061 every 8 connection(s)
I20260924 13:48:30.414746 23581 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790257629041645-13911-0/minicluster-data/ts-2/data/info.pb
I20260924 13:48:30.419759 23298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:48:30.415409 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.149.195:44493 (local address 127.13.149.254:44415)
0924 13:48:30.415465 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0924 13:48:30.415467 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:48:30.415707 (+ 240us) server_negotiation.cc:407] Connection header received
0924 13:48:30.416219 (+ 512us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:48:30.416221 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:48:30.416234 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:48:30.416285 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:48:30.416961 (+ 676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:48:30.417472 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:48:30.418439 (+ 967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:48:30.418637 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:48:30.418692 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:48:30.419495 (+ 803us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:48:30.419496 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:48:30.419496 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 13:48:30.419502 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:48:30.419518 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 13:48:30.419522 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:48:30.419564 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0924 13:48:30.419647 (+ 83us) server_negotiation.cc:299] Negotiation successful
0924 13:48:30.419674 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260924 13:48:30.419976 23711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:48:30.415415 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:44415 (local address 127.13.149.195:44493)
0924 13:48:30.415591 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0924 13:48:30.415625 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0924 13:48:30.416117 (+ 492us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:48:30.416351 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:48:30.416355 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:48:30.416363 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0924 13:48:30.416856 (+ 493us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:48:30.416861 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:48:30.417561 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:48:30.417561 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:48:30.418348 (+ 787us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:48:30.418351 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:48:30.418710 (+ 359us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:48:30.418711 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:48:30.418835 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:48:30.419343 (+ 508us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0924 13:48:30.419344 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0924 13:48:30.419372 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0924 13:48:30.419397 (+ 25us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 13:48:30.419575 (+ 178us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 13:48:30.419580 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 13:48:30.419583 (+ 3us) client_negotiation.cc:769] Sending connection context
0924 13:48:30.419644 (+ 61us) client_negotiation.cc:240] Negotiation successful
0924 13:48:30.419750 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":75,"threads_started":1}
I20260924 13:48:30.420459 23709 heartbeater.cc:347] Connected to a master server at 127.13.149.254:44415
I20260924 13:48:30.420542 23709 heartbeater.cc:464] Registering TS with master...
I20260924 13:48:30.420708 23709 heartbeater.cc:511] Master 127.13.149.254:44415 requested a full tablet report, sending...
I20260924 13:48:30.421085 23259 ts_manager.cc:194] Registered new tserver with Master: f163833346484109b7324f1d7aa92e2c (127.13.149.195:46061)
I20260924 13:48:30.421458 23259 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.13.149.195:44493
I20260924 13:48:30.423525 13911 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJN9WRF/build/release/bin/kudu as pid 23581
I20260924 13:48:30.423605 13911 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790257629041645-13911-0/minicluster-data/ts-2/wal/instance
I20260924 13:48:30.423784 13911 external_mini_cluster.cc:1428] Setting key eb2e1de3239c57f7456a5b35241d2f5f
I20260924 13:48:30.424700 13911 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260924 13:48:30.426450 23298 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 13:48:30.425991 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42522 (local address 127.13.149.254:44415)
0924 13:48:30.426052 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0924 13:48:30.426054 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:48:30.426121 (+ 67us) server_negotiation.cc:407] Connection header received
0924 13:48:30.426193 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:48:30.426194 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:48:30.426203 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:48:30.426249 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:48:30.426380 (+ 131us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42522: BlockingRecv error: recv got EOF from 127.0.0.1:42522 (error 108)
Metrics: {"server-negotiator.queue_time_us":35}
I20260924 13:48:30.426489 13911 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJN9WRF/build/release/bin/kudu with pid 23319
I20260924 13:48:30.430725 13911 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJN9WRF/build/release/bin/kudu with pid 23450
I20260924 13:48:30.435042 13911 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJN9WRF/build/release/bin/kudu with pid 23581
I20260924 13:48:30.438767 13911 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJN9WRF/build/release/bin/kudu with pid 23228
2026-09-24T13:48:30Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (576 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-09-24T13:48:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T13:48:30Z Disabled control of system clock
I20260924 13:48:30.463780 13911 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJN9WRF/build/release/bin/kudu
/tmp/dist-test-taskJN9WRF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790257629041645-13911-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790257629041645-13911-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790257629041645-13911-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790257629041645-13911-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.149.254:44835
--webserver_interface=127.13.149.254
--webserver_port=0
--builtin_ntp_servers=127.13.149.212:45115
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.149.254:44835
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260924 13:48:30.543217 23720 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 13:48:30.543453 23720 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 13:48:30.543475 23720 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 13:48:30.544603 23720 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 13:48:30.544647 23720 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 13:48:30.544659 23720 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 13:48:30.544670 23720 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 13:48:30.544680 23720 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 13:48:30.546082 23720 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.149.212:45115
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790257629041645-13911-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790257629041645-13911-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.149.254:44835
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.149.254:44835
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790257629041645-13911-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.149.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23720
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790257629041645-13911-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 12:29:37 UTC on ccycloud.quasar-iqzbxn.root.comops.site
build id 4670106
I20260924 13:48:30.546324 23720 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 13:48:30.546587 23720 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 13:48:30.551505 23726 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 13:48:30.551551 23720 server_base.cc:1034] running on GCE node
W20260924 13:48:30.551501 23725 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 13:48:30.551501 23728 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 13:48:30.552039 23720 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 13:48:30.552348 23720 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 13:48:30.553529 23720 hybrid_clock.cc:648] HybridClock initialized: now 1790257710553514 us; error 30 us; skew 500 ppm
I20260924 13:48:30.555017 23720 webserver.cc:460] Webserver started at http://127.13.149.254:33087/ using document root <none> and password file <none>
I20260924 13:48:30.555250 23720 fs_manager.cc:359] Metadata directory not provided
I20260924 13:48:30.555299 23720 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 13:48:30.555437 23720 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 13:48:30.556545 23720 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790257629041645-13911-0/minicluster-data/master-0/data/instance:
uuid: "d39d72b16b3648e28eae2126cc503099"
format_stamp: "Formatted at 2026-09-24 13:48:30 on dist-test-slave-sjl2"
server_key: "aa08b785900fefbc0e30717df2fdc1d8"
server_key_iv: "a2c19d26cbf0cab8e6ab5b344382f768"
server_key_version: "encryptionkey@0"
I20260924 13:48:30.556895 23720 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790257629041645-13911-0/minicluster-data/master-0/wal/instance:
uuid: "d39d72b16b3648e28eae2126cc503099"
format_stamp: "Formatted at 2026-09-24 13:48:30 on dist-test-slave-sjl2"
server_key: "aa08b785900fefbc0e30717df2fdc1d8"
server_key_iv: "a2c19d26cbf0cab8e6ab5b344382f768"
server_key_version: "encryptionkey@0"
I20260924 13:48:30.558269 23720 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260924 13:48:30.558977 23734 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 13:48:30.559135 23720 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 13:48:30.559216 23720 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790257629041645-13911-0/minicluster-data/master-0/data,/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790257629041645-13911-0/minicluster-data/master-0/wal
uuid: "d39d72b16b3648e28eae2126cc503099"
format_stamp: "Formatted at 2026-09-24 13:48:30 on dist-test-slave-sjl2"
server_key: "aa08b785900fefbc0e30717df2fdc1d8"
server_key_iv: "a2c19d26cbf0cab8e6ab5b344382f768"
server_key_version: "encryptionkey@0"
I20260924 13:48:30.559284 23720 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790257629041645-13911-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790257629041645-13911-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790257629041645-13911-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 13:48:30.562623 23720 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 13:48:30.562857 23720 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 13:48:30.562979 23720 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 13:48:30.566598 23720 rpc_server.cc:307] RPC server started. Bound to: 127.13.149.254:44835
I20260924 13:48:30.566632 23786 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.149.254:44835 every 8 connection(s)
I20260924 13:48:30.566946 23720 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790257629041645-13911-0/minicluster-data/master-0/data/info.pb
I20260924 13:48:30.567605 23787 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 13:48:30.567741 13911 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJN9WRF/build/release/bin/kudu as pid 23720
I20260924 13:48:30.567831 13911 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790257629041645-13911-0/minicluster-data/master-0/wal/instance
I20260924 13:48:30.568011 13911 external_mini_cluster.cc:1428] Setting key 80229dafba25c596241a5b57d8d7ebf2
I20260924 13:48:30.570573 23790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:48:30.568620 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49636 (local address 127.13.149.254:44835)
0924 13:48:30.568829 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0924 13:48:30.568838 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0924 13:48:30.568871 (+ 33us) server_negotiation.cc:407] Connection header received
0924 13:48:30.569319 (+ 448us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:48:30.569328 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:48:30.569391 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:48:30.569557 (+ 166us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:48:30.570150 (+ 593us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:48:30.570154 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:48:30.570155 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 13:48:30.570159 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:48:30.570206 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 13:48:30.570220 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:48:30.570290 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0924 13:48:30.570326 (+ 36us) server_negotiation.cc:299] Negotiation successful
0924 13:48:30.570383 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":132,"thread_start_us":87,"threads_started":1}
I20260924 13:48:30.571377 23787 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d39d72b16b3648e28eae2126cc503099: Bootstrap starting.
I20260924 13:48:30.571941 23787 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d39d72b16b3648e28eae2126cc503099: Neither blocks nor log segments found. Creating new log.
I20260924 13:48:30.572198 23787 log.cc:824] T 00000000000000000000000000000000 P d39d72b16b3648e28eae2126cc503099: Log is configured to *not* fsync() on all Append() calls
I20260924 13:48:30.572849 23787 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d39d72b16b3648e28eae2126cc503099: No bootstrap required, opened a new log
I20260924 13:48:30.574252 23787 raft_consensus.cc:348] T 00000000000000000000000000000000 P d39d72b16b3648e28eae2126cc503099 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d39d72b16b3648e28eae2126cc503099" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 44835 } }
I20260924 13:48:30.574394 23787 raft_consensus.cc:374] T 00000000000000000000000000000000 P d39d72b16b3648e28eae2126cc503099 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 13:48:30.574417 23787 raft_consensus.cc:729] T 00000000000000000000000000000000 P d39d72b16b3648e28eae2126cc503099 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d39d72b16b3648e28eae2126cc503099, State: Initialized, Role: FOLLOWER
I20260924 13:48:30.574530 23787 consensus_queue.cc:260] T 00000000000000000000000000000000 P d39d72b16b3648e28eae2126cc503099 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d39d72b16b3648e28eae2126cc503099" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 44835 } }
I20260924 13:48:30.574604 23787 raft_consensus.cc:388] T 00000000000000000000000000000000 P d39d72b16b3648e28eae2126cc503099 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 13:48:30.574630 23787 raft_consensus.cc:482] T 00000000000000000000000000000000 P d39d72b16b3648e28eae2126cc503099 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 13:48:30.574679 23787 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d39d72b16b3648e28eae2126cc503099 [term 0 FOLLOWER]: Advancing to term 1
I20260924 13:48:30.575217 23787 raft_consensus.cc:504] T 00000000000000000000000000000000 P d39d72b16b3648e28eae2126cc503099 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d39d72b16b3648e28eae2126cc503099" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 44835 } }
I20260924 13:48:30.575312 23787 leader_election.cc:302] T 00000000000000000000000000000000 P d39d72b16b3648e28eae2126cc503099 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d39d72b16b3648e28eae2126cc503099; no voters:
I20260924 13:48:30.575460 23787 leader_election.cc:288] T 00000000000000000000000000000000 P d39d72b16b3648e28eae2126cc503099 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 13:48:30.575690 23787 sys_catalog.cc:564] T 00000000000000000000000000000000 P d39d72b16b3648e28eae2126cc503099 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 13:48:30.575770 23792 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d39d72b16b3648e28eae2126cc503099 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 13:48:30.575910 23792 raft_consensus.cc:686] T 00000000000000000000000000000000 P d39d72b16b3648e28eae2126cc503099 [term 1 LEADER]: Becoming Leader. State: Replica: d39d72b16b3648e28eae2126cc503099, State: Running, Role: LEADER
I20260924 13:48:30.576048 23792 consensus_queue.cc:237] T 00000000000000000000000000000000 P d39d72b16b3648e28eae2126cc503099 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d39d72b16b3648e28eae2126cc503099" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 44835 } }
I20260924 13:48:30.576432 23794 sys_catalog.cc:455] T 00000000000000000000000000000000 P d39d72b16b3648e28eae2126cc503099 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d39d72b16b3648e28eae2126cc503099. Latest consensus state: current_term: 1 leader_uuid: "d39d72b16b3648e28eae2126cc503099" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d39d72b16b3648e28eae2126cc503099" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 44835 } } }
I20260924 13:48:30.576545 23794 sys_catalog.cc:458] T 00000000000000000000000000000000 P d39d72b16b3648e28eae2126cc503099 [sys.catalog]: This master's current role is: LEADER
I20260924 13:48:30.576422 23793 sys_catalog.cc:455] T 00000000000000000000000000000000 P d39d72b16b3648e28eae2126cc503099 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d39d72b16b3648e28eae2126cc503099" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d39d72b16b3648e28eae2126cc503099" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 44835 } } }
I20260924 13:48:30.576691 23793 sys_catalog.cc:458] T 00000000000000000000000000000000 P d39d72b16b3648e28eae2126cc503099 [sys.catalog]: This master's current role is: LEADER
I20260924 13:48:30.576979 23802 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 13:48:30.577533 23802 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 13:48:30.579030 23802 catalog_manager.cc:1347] Generated new cluster ID: 63eb918393294651ad3d05ae21a2a1a8
I20260924 13:48:30.579110 23802 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 13:48:30.603219 23802 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 13:48:30.603420 23802 catalog_manager.cc:1504] Loading token signing keys...
I20260924 13:48:30.608634 23802 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d39d72b16b3648e28eae2126cc503099: Generated new TSK 0
I20260924 13:48:30.608841 23802 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260924 13:48:30.612854 13911 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJN9WRF/build/release/bin/kudu
/tmp/dist-test-taskJN9WRF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790257629041645-13911-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790257629041645-13911-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790257629041645-13911-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790257629041645-13911-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.149.193:0
--local_ip_for_outbound_sockets=127.13.149.193
--webserver_interface=127.13.149.193
--webserver_port=0
--tserver_master_addrs=127.13.149.254:44835
--builtin_ntp_servers=127.13.149.212:45115
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260924 13:48:30.693450 23811 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 13:48:30.693650 23811 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 13:48:30.693671 23811 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 13:48:30.694851 23811 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 13:48:30.694891 23811 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 13:48:30.694917 23811 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.149.193
I20260924 13:48:30.696269 23811 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.149.212:45115
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790257629041645-13911-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790257629041645-13911-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.149.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790257629041645-13911-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.149.193
--webserver_port=0
--tserver_master_addrs=127.13.149.254:44835
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23811
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.149.193
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790257629041645-13911-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 12:29:37 UTC on ccycloud.quasar-iqzbxn.root.comops.site
build id 4670106
I20260924 13:48:30.696504 23811 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 13:48:30.696808 23811 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 13:48:30.701637 23817 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 13:48:30.701642 23816 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 13:48:30.701673 23811 server_base.cc:1034] running on GCE node
W20260924 13:48:30.701658 23819 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 13:48:30.702116 23811 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 13:48:30.702330 23811 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 13:48:30.703472 23811 hybrid_clock.cc:648] HybridClock initialized: now 1790257710703459 us; error 30 us; skew 500 ppm
I20260924 13:48:30.704725 23811 webserver.cc:460] Webserver started at http://127.13.149.193:34087/ using document root <none> and password file <none>
I20260924 13:48:30.704984 23811 fs_manager.cc:359] Metadata directory not provided
I20260924 13:48:30.705034 23811 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 13:48:30.705154 23811 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 13:48:30.706212 23811 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790257629041645-13911-0/minicluster-data/ts-0/data/instance:
uuid: "48f9d9672dd44c53b7b4e2c9cdeff44a"
format_stamp: "Formatted at 2026-09-24 13:48:30 on dist-test-slave-sjl2"
server_key: "12b0f05f8fbc785cf0f92565979a7748"
server_key_iv: "d7e3569036e99dc73ec62eed5f672755"
server_key_version: "encryptionkey@0"
I20260924 13:48:30.706535 23811 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790257629041645-13911-0/minicluster-data/ts-0/wal/instance:
uuid: "48f9d9672dd44c53b7b4e2c9cdeff44a"
format_stamp: "Formatted at 2026-09-24 13:48:30 on dist-test-slave-sjl2"
server_key: "12b0f05f8fbc785cf0f92565979a7748"
server_key_iv: "d7e3569036e99dc73ec62eed5f672755"
server_key_version: "encryptionkey@0"
I20260924 13:48:30.707715 23811 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260924 13:48:30.708441 23825 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 13:48:30.708715 23811 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260924 13:48:30.708820 23811 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790257629041645-13911-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790257629041645-13911-0/minicluster-data/ts-0/wal
uuid: "48f9d9672dd44c53b7b4e2c9cdeff44a"
format_stamp: "Formatted at 2026-09-24 13:48:30 on dist-test-slave-sjl2"
server_key: "12b0f05f8fbc785cf0f92565979a7748"
server_key_iv: "d7e3569036e99dc73ec62eed5f672755"
server_key_version: "encryptionkey@0"
I20260924 13:48:30.708901 23811 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790257629041645-13911-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790257629041645-13911-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790257629041645-13911-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 13:48:30.712353 23811 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 13:48:30.712637 23811 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 13:48:30.712769 23811 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 13:48:30.713001 23811 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 13:48:30.713399 23811 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 13:48:30.713486 23811 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 13:48:30.713523 23811 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 13:48:30.713552 23811 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 13:48:30.719178 23811 rpc_server.cc:307] RPC server started. Bound to: 127.13.149.193:34191
I20260924 13:48:30.719250 23938 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.149.193:34191 every 8 connection(s)
I20260924 13:48:30.719580 23811 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790257629041645-13911-0/minicluster-data/ts-0/data/info.pb
I20260924 13:48:30.722441 23790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:48:30.720313 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.149.193:36001 (local address 127.13.149.254:44835)
0924 13:48:30.720399 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0924 13:48:30.720401 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:48:30.720579 (+ 178us) server_negotiation.cc:407] Connection header received
0924 13:48:30.721120 (+ 541us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:48:30.721122 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:48:30.721128 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:48:30.721169 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:48:30.722250 (+ 1081us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:48:30.722252 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:48:30.722252 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 13:48:30.722256 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:48:30.722271 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 13:48:30.722275 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:48:30.722319 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0924 13:48:30.722370 (+ 51us) server_negotiation.cc:299] Negotiation successful
0924 13:48:30.722383 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260924 13:48:30.722625 23941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:48:30.720279 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:44835 (local address 127.13.149.193:36001)
0924 13:48:30.720465 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0924 13:48:30.720500 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0924 13:48:30.721017 (+ 517us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:48:30.721247 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:48:30.721251 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:48:30.721260 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0924 13:48:30.722109 (+ 849us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0924 13:48:30.722110 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0924 13:48:30.722140 (+ 30us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0924 13:48:30.722169 (+ 29us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 13:48:30.722318 (+ 149us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 13:48:30.722320 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 13:48:30.722322 (+ 2us) client_negotiation.cc:769] Sending connection context
0924 13:48:30.722371 (+ 49us) client_negotiation.cc:240] Negotiation successful
0924 13:48:30.722421 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":83,"threads_started":1}
I20260924 13:48:30.723121 23939 heartbeater.cc:347] Connected to a master server at 127.13.149.254:44835
I20260924 13:48:30.723207 23939 heartbeater.cc:464] Registering TS with master...
I20260924 13:48:30.723351 23939 heartbeater.cc:511] Master 127.13.149.254:44835 requested a full tablet report, sending...
I20260924 13:48:30.723860 23751 ts_manager.cc:194] Registered new tserver with Master: 48f9d9672dd44c53b7b4e2c9cdeff44a (127.13.149.193:34191)
I20260924 13:48:30.727380 13911 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJN9WRF/build/release/bin/kudu as pid 23811
I20260924 13:48:30.727485 13911 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790257629041645-13911-0/minicluster-data/ts-0/wal/instance
I20260924 13:48:30.727686 13911 external_mini_cluster.cc:1428] Setting key 389ada75a5965276dad30f4fbdb05d62
I20260924 13:48:30.728650 13911 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJN9WRF/build/release/bin/kudu
/tmp/dist-test-taskJN9WRF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790257629041645-13911-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790257629041645-13911-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790257629041645-13911-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790257629041645-13911-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.149.194:0
--local_ip_for_outbound_sockets=127.13.149.194
--webserver_interface=127.13.149.194
--webserver_port=0
--tserver_master_addrs=127.13.149.254:44835
--builtin_ntp_servers=127.13.149.212:45115
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260924 13:48:30.808779 23942 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 13:48:30.809024 23942 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 13:48:30.809048 23942 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 13:48:30.810268 23942 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 13:48:30.810318 23942 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 13:48:30.810348 23942 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.149.194
I20260924 13:48:30.811782 23942 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.149.212:45115
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790257629041645-13911-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790257629041645-13911-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.149.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790257629041645-13911-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.149.194
--webserver_port=0
--tserver_master_addrs=127.13.149.254:44835
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23942
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.149.194
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790257629041645-13911-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 12:29:37 UTC on ccycloud.quasar-iqzbxn.root.comops.site
build id 4670106
I20260924 13:48:30.812029 23942 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 13:48:30.812330 23942 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260924 13:48:30.817502 23942 server_base.cc:1034] running on GCE node
W20260924 13:48:30.817589 23948 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 13:48:30.817554 23950 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 13:48:30.817502 23947 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 13:48:30.817873 23942 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 13:48:30.818090 23942 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 13:48:30.819240 23942 hybrid_clock.cc:648] HybridClock initialized: now 1790257710819226 us; error 29 us; skew 500 ppm
I20260924 13:48:30.820528 23942 webserver.cc:460] Webserver started at http://127.13.149.194:37225/ using document root <none> and password file <none>
I20260924 13:48:30.820761 23942 fs_manager.cc:359] Metadata directory not provided
I20260924 13:48:30.820812 23942 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 13:48:30.820982 23942 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 13:48:30.822045 23942 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790257629041645-13911-0/minicluster-data/ts-1/data/instance:
uuid: "559de5e8493f4bfd98dea64dca49f640"
format_stamp: "Formatted at 2026-09-24 13:48:30 on dist-test-slave-sjl2"
server_key: "129070985f69dfa0c86ac7429c55d11f"
server_key_iv: "d547a84ab3c31743cfa573fcd6e46957"
server_key_version: "encryptionkey@0"
I20260924 13:48:30.822376 23942 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790257629041645-13911-0/minicluster-data/ts-1/wal/instance:
uuid: "559de5e8493f4bfd98dea64dca49f640"
format_stamp: "Formatted at 2026-09-24 13:48:30 on dist-test-slave-sjl2"
server_key: "129070985f69dfa0c86ac7429c55d11f"
server_key_iv: "d547a84ab3c31743cfa573fcd6e46957"
server_key_version: "encryptionkey@0"
I20260924 13:48:30.823654 23942 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260924 13:48:30.824380 23956 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 13:48:30.824566 23942 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 13:48:30.824649 23942 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790257629041645-13911-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790257629041645-13911-0/minicluster-data/ts-1/wal
uuid: "559de5e8493f4bfd98dea64dca49f640"
format_stamp: "Formatted at 2026-09-24 13:48:30 on dist-test-slave-sjl2"
server_key: "129070985f69dfa0c86ac7429c55d11f"
server_key_iv: "d547a84ab3c31743cfa573fcd6e46957"
server_key_version: "encryptionkey@0"
I20260924 13:48:30.824723 23942 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790257629041645-13911-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790257629041645-13911-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790257629041645-13911-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 13:48:30.828147 23942 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 13:48:30.828415 23942 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 13:48:30.828543 23942 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 13:48:30.828732 23942 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 13:48:30.829056 23942 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 13:48:30.829092 23942 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 13:48:30.829140 23942 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 13:48:30.829164 23942 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 13:48:30.834579 23942 rpc_server.cc:307] RPC server started. Bound to: 127.13.149.194:34121
I20260924 13:48:30.834642 24069 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.149.194:34121 every 8 connection(s)
I20260924 13:48:30.834946 23942 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790257629041645-13911-0/minicluster-data/ts-1/data/info.pb
I20260924 13:48:30.837849 23790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:48:30.835630 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.149.194:59409 (local address 127.13.149.254:44835)
0924 13:48:30.835695 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0924 13:48:30.835697 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:48:30.835899 (+ 202us) server_negotiation.cc:407] Connection header received
0924 13:48:30.836403 (+ 504us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:48:30.836405 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:48:30.836412 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:48:30.836465 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:48:30.837613 (+ 1148us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:48:30.837615 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:48:30.837616 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 13:48:30.837620 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:48:30.837636 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 13:48:30.837640 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:48:30.837690 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0924 13:48:30.837761 (+ 71us) server_negotiation.cc:299] Negotiation successful
0924 13:48:30.837777 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260924 13:48:30.838025 24072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:48:30.835602 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:44835 (local address 127.13.149.194:59409)
0924 13:48:30.835783 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0924 13:48:30.835817 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0924 13:48:30.836301 (+ 484us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:48:30.836535 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:48:30.836539 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:48:30.836548 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0924 13:48:30.837458 (+ 910us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0924 13:48:30.837459 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0924 13:48:30.837488 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0924 13:48:30.837519 (+ 31us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 13:48:30.837694 (+ 175us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 13:48:30.837699 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 13:48:30.837702 (+ 3us) client_negotiation.cc:769] Sending connection context
0924 13:48:30.837768 (+ 66us) client_negotiation.cc:240] Negotiation successful
0924 13:48:30.837829 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":80,"threads_started":1}
I20260924 13:48:30.838474 24070 heartbeater.cc:347] Connected to a master server at 127.13.149.254:44835
I20260924 13:48:30.838567 24070 heartbeater.cc:464] Registering TS with master...
I20260924 13:48:30.838709 24070 heartbeater.cc:511] Master 127.13.149.254:44835 requested a full tablet report, sending...
I20260924 13:48:30.839134 23751 ts_manager.cc:194] Registered new tserver with Master: 559de5e8493f4bfd98dea64dca49f640 (127.13.149.194:34121)
I20260924 13:48:30.843251 13911 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJN9WRF/build/release/bin/kudu as pid 23942
I20260924 13:48:30.843344 13911 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790257629041645-13911-0/minicluster-data/ts-1/wal/instance
I20260924 13:48:30.843556 13911 external_mini_cluster.cc:1428] Setting key 38ba5ab27543f58ae240ed68b67ffb35
I20260924 13:48:30.844462 13911 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJN9WRF/build/release/bin/kudu
/tmp/dist-test-taskJN9WRF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790257629041645-13911-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790257629041645-13911-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790257629041645-13911-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790257629041645-13911-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.149.195:0
--local_ip_for_outbound_sockets=127.13.149.195
--webserver_interface=127.13.149.195
--webserver_port=0
--tserver_master_addrs=127.13.149.254:44835
--builtin_ntp_servers=127.13.149.212:45115
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260924 13:48:30.924510 24073 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 13:48:30.924742 24073 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 13:48:30.924764 24073 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 13:48:30.926020 24073 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 13:48:30.926066 24073 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 13:48:30.926093 24073 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.149.195
I20260924 13:48:30.927488 24073 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.149.212:45115
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790257629041645-13911-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790257629041645-13911-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.149.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790257629041645-13911-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.149.195
--webserver_port=0
--tserver_master_addrs=127.13.149.254:44835
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24073
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.149.195
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790257629041645-13911-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 12:29:37 UTC on ccycloud.quasar-iqzbxn.root.comops.site
build id 4670106
I20260924 13:48:30.927747 24073 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 13:48:30.928028 24073 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 13:48:30.933130 24079 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 13:48:30.933143 24078 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 13:48:30.933130 24081 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 13:48:30.933386 24073 server_base.cc:1034] running on GCE node
I20260924 13:48:30.933646 24073 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 13:48:30.933883 24073 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 13:48:30.935047 24073 hybrid_clock.cc:648] HybridClock initialized: now 1790257710935024 us; error 42 us; skew 500 ppm
I20260924 13:48:30.939644 24073 webserver.cc:460] Webserver started at http://127.13.149.195:41711/ using document root <none> and password file <none>
I20260924 13:48:30.939869 24073 fs_manager.cc:359] Metadata directory not provided
I20260924 13:48:30.939919 24073 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 13:48:30.940050 24073 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 13:48:30.941192 24073 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790257629041645-13911-0/minicluster-data/ts-2/data/instance:
uuid: "5771ddca71914773b2f92973c0ae850b"
format_stamp: "Formatted at 2026-09-24 13:48:30 on dist-test-slave-sjl2"
server_key: "b42d50ed3367be69ae15b7ff65928ebb"
server_key_iv: "cbac83948b6d4d7c83ad8a4684108516"
server_key_version: "encryptionkey@0"
I20260924 13:48:30.941533 24073 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790257629041645-13911-0/minicluster-data/ts-2/wal/instance:
uuid: "5771ddca71914773b2f92973c0ae850b"
format_stamp: "Formatted at 2026-09-24 13:48:30 on dist-test-slave-sjl2"
server_key: "b42d50ed3367be69ae15b7ff65928ebb"
server_key_iv: "cbac83948b6d4d7c83ad8a4684108516"
server_key_version: "encryptionkey@0"
I20260924 13:48:30.942741 24073 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260924 13:48:30.943395 24087 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 13:48:30.943562 24073 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 13:48:30.943645 24073 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790257629041645-13911-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790257629041645-13911-0/minicluster-data/ts-2/wal
uuid: "5771ddca71914773b2f92973c0ae850b"
format_stamp: "Formatted at 2026-09-24 13:48:30 on dist-test-slave-sjl2"
server_key: "b42d50ed3367be69ae15b7ff65928ebb"
server_key_iv: "cbac83948b6d4d7c83ad8a4684108516"
server_key_version: "encryptionkey@0"
I20260924 13:48:30.943706 24073 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790257629041645-13911-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790257629041645-13911-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790257629041645-13911-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 13:48:30.947023 24073 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 13:48:30.947264 24073 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 13:48:30.947382 24073 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 13:48:30.947574 24073 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 13:48:30.947855 24073 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 13:48:30.947887 24073 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 13:48:30.947912 24073 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 13:48:30.947937 24073 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 13:48:30.953488 24073 rpc_server.cc:307] RPC server started. Bound to: 127.13.149.195:46309
I20260924 13:48:30.953540 24200 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.149.195:46309 every 8 connection(s)
I20260924 13:48:30.953860 24073 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790257629041645-13911-0/minicluster-data/ts-2/data/info.pb
I20260924 13:48:30.956710 23790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:48:30.954572 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.149.195:34357 (local address 127.13.149.254:44835)
0924 13:48:30.954647 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0924 13:48:30.954650 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 13:48:30.954827 (+ 177us) server_negotiation.cc:407] Connection header received
0924 13:48:30.955299 (+ 472us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:48:30.955301 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:48:30.955306 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:48:30.955367 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:48:30.956506 (+ 1139us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:48:30.956508 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:48:30.956508 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 13:48:30.956512 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:48:30.956527 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 13:48:30.956531 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:48:30.956579 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0924 13:48:30.956645 (+ 66us) server_negotiation.cc:299] Negotiation successful
0924 13:48:30.956656 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260924 13:48:30.956885 24203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:48:30.954539 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:44835 (local address 127.13.149.195:34357)
0924 13:48:30.954711 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0924 13:48:30.954746 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0924 13:48:30.955210 (+ 464us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:48:30.955442 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:48:30.955445 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:48:30.955454 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0924 13:48:30.956354 (+ 900us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0924 13:48:30.956354 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0924 13:48:30.956386 (+ 32us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0924 13:48:30.956414 (+ 28us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 13:48:30.956587 (+ 173us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 13:48:30.956590 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 13:48:30.956592 (+ 2us) client_negotiation.cc:769] Sending connection context
0924 13:48:30.956649 (+ 57us) client_negotiation.cc:240] Negotiation successful
0924 13:48:30.956700 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":77,"threads_started":1}
I20260924 13:48:30.957345 24201 heartbeater.cc:347] Connected to a master server at 127.13.149.254:44835
I20260924 13:48:30.957440 24201 heartbeater.cc:464] Registering TS with master...
I20260924 13:48:30.957587 24201 heartbeater.cc:511] Master 127.13.149.254:44835 requested a full tablet report, sending...
I20260924 13:48:30.957978 23751 ts_manager.cc:194] Registered new tserver with Master: 5771ddca71914773b2f92973c0ae850b (127.13.149.195:46309)
I20260924 13:48:30.958865 13911 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJN9WRF/build/release/bin/kudu as pid 24073
I20260924 13:48:30.958945 13911 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790257629041645-13911-0/minicluster-data/ts-2/wal/instance
I20260924 13:48:30.959100 13911 external_mini_cluster.cc:1428] Setting key 9e077ac7194d9443843f9dd54fb8a491
I20260924 13:48:30.959794 13911 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260924 13:48:30.961598 13911 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJN9WRF/build/release/bin/kudu with pid 23811
W20260924 13:48:30.961644 23790 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 13:48:30.961124 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49650 (local address 127.13.149.254:44835)
0924 13:48:30.961190 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0924 13:48:30.961193 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 13:48:30.961269 (+ 76us) server_negotiation.cc:407] Connection header received
0924 13:48:30.961339 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:48:30.961339 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:48:30.961344 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:48:30.961380 (+ 36us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:48:30.961550 (+ 170us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49650: BlockingRecv error: recv got EOF from 127.0.0.1:49650 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
I20260924 13:48:30.967275 13911 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJN9WRF/build/release/bin/kudu with pid 23942
I20260924 13:48:30.971221 13911 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJN9WRF/build/release/bin/kudu with pid 24073
I20260924 13:48:30.975322 13911 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJN9WRF/build/release/bin/kudu with pid 23720
2026-09-24T13:48:30Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (536 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790257629041645-13911-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 13:48:31 dist-test-slave-sjl2 krb5kdc[24211](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 13:48:31 dist-test-slave-sjl2 krb5kdc[24211](info): set up 2 sockets
Sep 24 13:48:31 dist-test-slave-sjl2 krb5kdc[24211](info): commencing operation
krb5kdc: starting...
W20260924 13:48:31.934391 13911 mini_kdc.cc:121] Time spent starting KDC: real 0.940s user 0.001s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790257629041645-13911-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790257629041645-13911-0/krb5kdc/test-user.keytab.
Sep 24 13:48:32 dist-test-slave-sjl2 krb5kdc[24211](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257712, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T13:48:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T13:48:32Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.149.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.149.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.149.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.254.keytab.
Entry for principal kudu/127.13.149.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.254.keytab.
I20260924 13:48:32.052598 13911 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJN9WRF/build/release/bin/kudu
/tmp/dist-test-taskJN9WRF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790257629041645-13911-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790257629041645-13911-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790257629041645-13911-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790257629041645-13911-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.149.254:36833
--webserver_interface=127.13.149.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.254.keytab
--principal=kudu/127.13.149.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.149.212:33805
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.149.254:36833
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790257629041645-13911-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790257629041645-13911-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790257629041645-13911-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 13:48:32.132131 24224 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 13:48:32.132359 24224 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 13:48:32.132390 24224 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 13:48:32.133641 24224 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 13:48:32.133684 24224 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 13:48:32.133703 24224 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 13:48:32.133720 24224 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 13:48:32.133735 24224 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 13:48:32.135285 24224 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.149.212:33805
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790257629041645-13911-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790257629041645-13911-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.149.254:36833
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.254.keytab
--principal=kudu/127.13.149.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.149.254:36833
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790257629041645-13911-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.149.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24224
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790257629041645-13911-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 12:29:37 UTC on ccycloud.quasar-iqzbxn.root.comops.site
build id 4670106
I20260924 13:48:32.135540 24224 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 13:48:32.135749 24224 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 13:48:32.140535 24229 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 13:48:32.140555 24224 server_base.cc:1034] running on GCE node
W20260924 13:48:32.140545 24230 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 13:48:32.140514 24232 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 13:48:32.141062 24224 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 13:48:32.141353 24224 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 13:48:32.142489 24224 hybrid_clock.cc:648] HybridClock initialized: now 1790257712142471 us; error 38 us; skew 500 ppm
Sep 24 13:48:32 dist-test-slave-sjl2 krb5kdc[24211](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257712, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.149.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 13:48:32.145191 24224 init.cc:377] Logged in from keytab as kudu/127.13.149.254@KRBTEST.COM (short username kudu)
I20260924 13:48:32.145630 24224 webserver.cc:460] Webserver started at http://127.13.149.254:34193/ using document root <none> and password file <none>
I20260924 13:48:32.145829 24224 fs_manager.cc:359] Metadata directory not provided
I20260924 13:48:32.145874 24224 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 13:48:32.146005 24224 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 13:48:32.147106 24224 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790257629041645-13911-0/minicluster-data/master-0/data/instance:
uuid: "1dd3595bceea42e385bf25a41f15e80a"
format_stamp: "Formatted at 2026-09-24 13:48:32 on dist-test-slave-sjl2"
server_key: "e44acac7d6280bd7b94710df6851b7c5"
server_key_iv: "8467c9ee29157937b94c82c09e193bb0"
server_key_version: "encryptionkey@0"
I20260924 13:48:32.147440 24224 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790257629041645-13911-0/minicluster-data/master-0/wal/instance:
uuid: "1dd3595bceea42e385bf25a41f15e80a"
format_stamp: "Formatted at 2026-09-24 13:48:32 on dist-test-slave-sjl2"
server_key: "e44acac7d6280bd7b94710df6851b7c5"
server_key_iv: "8467c9ee29157937b94c82c09e193bb0"
server_key_version: "encryptionkey@0"
I20260924 13:48:32.148723 24224 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260924 13:48:32.149564 24239 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 13:48:32.149720 24224 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260924 13:48:32.149806 24224 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790257629041645-13911-0/minicluster-data/master-0/data,/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790257629041645-13911-0/minicluster-data/master-0/wal
uuid: "1dd3595bceea42e385bf25a41f15e80a"
format_stamp: "Formatted at 2026-09-24 13:48:32 on dist-test-slave-sjl2"
server_key: "e44acac7d6280bd7b94710df6851b7c5"
server_key_iv: "8467c9ee29157937b94c82c09e193bb0"
server_key_version: "encryptionkey@0"
I20260924 13:48:32.149874 24224 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790257629041645-13911-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790257629041645-13911-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790257629041645-13911-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 13:48:32.158281 24224 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 13:48:32.160317 24224 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 13:48:32.160463 24224 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 13:48:32.164541 24224 rpc_server.cc:307] RPC server started. Bound to: 127.13.149.254:36833
I20260924 13:48:32.164573 24291 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.149.254:36833 every 8 connection(s)
I20260924 13:48:32.164911 24224 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790257629041645-13911-0/minicluster-data/master-0/data/info.pb
I20260924 13:48:32.165971 24292 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 13:48:32.167048 13911 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJN9WRF/build/release/bin/kudu as pid 24224
I20260924 13:48:32.167156 13911 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790257629041645-13911-0/minicluster-data/master-0/wal/instance
I20260924 13:48:32.167346 13911 external_mini_cluster.cc:1428] Setting key ce60e0edfc0221fd936d3af5427b9def
I20260924 13:48:32.168737 24292 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1dd3595bceea42e385bf25a41f15e80a: Bootstrap starting.
I20260924 13:48:32.169359 24292 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1dd3595bceea42e385bf25a41f15e80a: Neither blocks nor log segments found. Creating new log.
I20260924 13:48:32.169626 24292 log.cc:824] T 00000000000000000000000000000000 P 1dd3595bceea42e385bf25a41f15e80a: Log is configured to *not* fsync() on all Append() calls
I20260924 13:48:32.170246 24292 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1dd3595bceea42e385bf25a41f15e80a: No bootstrap required, opened a new log
I20260924 13:48:32.171988 24292 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1dd3595bceea42e385bf25a41f15e80a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1dd3595bceea42e385bf25a41f15e80a" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 36833 } }
I20260924 13:48:32.172128 24292 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1dd3595bceea42e385bf25a41f15e80a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 13:48:32.172159 24292 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1dd3595bceea42e385bf25a41f15e80a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1dd3595bceea42e385bf25a41f15e80a, State: Initialized, Role: FOLLOWER
I20260924 13:48:32.172298 24292 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1dd3595bceea42e385bf25a41f15e80a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1dd3595bceea42e385bf25a41f15e80a" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 36833 } }
I20260924 13:48:32.172381 24292 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1dd3595bceea42e385bf25a41f15e80a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 13:48:32.172405 24292 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1dd3595bceea42e385bf25a41f15e80a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 13:48:32.172451 24292 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1dd3595bceea42e385bf25a41f15e80a [term 0 FOLLOWER]: Advancing to term 1
I20260924 13:48:32.173025 24292 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1dd3595bceea42e385bf25a41f15e80a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1dd3595bceea42e385bf25a41f15e80a" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 36833 } }
I20260924 13:48:32.173126 24292 leader_election.cc:302] T 00000000000000000000000000000000 P 1dd3595bceea42e385bf25a41f15e80a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1dd3595bceea42e385bf25a41f15e80a; no voters:
I20260924 13:48:32.173318 24292 leader_election.cc:288] T 00000000000000000000000000000000 P 1dd3595bceea42e385bf25a41f15e80a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 13:48:32.173357 24297 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1dd3595bceea42e385bf25a41f15e80a [term 1 FOLLOWER]: Leader election won for term 1
I20260924 13:48:32.173533 24297 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1dd3595bceea42e385bf25a41f15e80a [term 1 LEADER]: Becoming Leader. State: Replica: 1dd3595bceea42e385bf25a41f15e80a, State: Running, Role: LEADER
I20260924 13:48:32.173575 24292 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1dd3595bceea42e385bf25a41f15e80a [sys.catalog]: configured and running, proceeding with master startup.
Sep 24 13:48:32 dist-test-slave-sjl2 krb5kdc[24211](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257712, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.149.254@KRBTEST.COM
I20260924 13:48:32.173684 24297 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1dd3595bceea42e385bf25a41f15e80a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1dd3595bceea42e385bf25a41f15e80a" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 36833 } }
I20260924 13:48:32.174306 24298 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1dd3595bceea42e385bf25a41f15e80a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1dd3595bceea42e385bf25a41f15e80a. Latest consensus state: current_term: 1 leader_uuid: "1dd3595bceea42e385bf25a41f15e80a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1dd3595bceea42e385bf25a41f15e80a" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 36833 } } }
I20260924 13:48:32.174511 24298 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1dd3595bceea42e385bf25a41f15e80a [sys.catalog]: This master's current role is: LEADER
I20260924 13:48:32.174317 24299 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1dd3595bceea42e385bf25a41f15e80a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1dd3595bceea42e385bf25a41f15e80a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1dd3595bceea42e385bf25a41f15e80a" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 36833 } } }
I20260924 13:48:32.174786 24299 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1dd3595bceea42e385bf25a41f15e80a [sys.catalog]: This master's current role is: LEADER
I20260924 13:48:32.174878 24306 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 13:48:32.175374 24306 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 13:48:32.176971 24306 catalog_manager.cc:1347] Generated new cluster ID: e46a629a8b034c7dbf43537c8d0d9e0f
I20260924 13:48:32.177018 24306 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 13:48:32.177722 24295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:48:32.167881 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51508 (local address 127.13.149.254:36833)
0924 13:48:32.168050 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0924 13:48:32.168060 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0924 13:48:32.168248 (+ 188us) server_negotiation.cc:407] Connection header received
0924 13:48:32.168714 (+ 466us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:48:32.168721 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:48:32.168769 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:48:32.168909 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:48:32.169550 (+ 641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:48:32.170510 (+ 960us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:48:32.171201 (+ 691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:48:32.171616 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:48:32.171710 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:48:32.174244 (+ 2534us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:48:32.174248 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:48:32.174249 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 13:48:32.174261 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:48:32.176015 (+ 1754us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:48:32.176340 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:48:32.176344 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:48:32.176344 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:48:32.176468 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:48:32.176648 (+ 180us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:48:32.176649 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:48:32.176650 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:48:32.176999 (+ 349us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:48:32.177131 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0924 13:48:32.177372 (+ 241us) server_negotiation.cc:299] Negotiation successful
0924 13:48:32.177518 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":118,"thread_start_us":66,"threads_started":1}
I20260924 13:48:32.186347 24306 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 13:48:32.187239 24306 catalog_manager.cc:1504] Loading token signing keys...
I20260924 13:48:32.196979 24306 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1dd3595bceea42e385bf25a41f15e80a: Generated new TSK 0
I20260924 13:48:32.197188 24306 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.13.149.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.149.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.149.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.193.keytab.
Entry for principal kudu/127.13.149.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.193.keytab.
I20260924 13:48:32.226166 13911 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJN9WRF/build/release/bin/kudu
/tmp/dist-test-taskJN9WRF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790257629041645-13911-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790257629041645-13911-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790257629041645-13911-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790257629041645-13911-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.149.193:0
--local_ip_for_outbound_sockets=127.13.149.193
--webserver_interface=127.13.149.193
--webserver_port=0
--tserver_master_addrs=127.13.149.254:36833
--keytab_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.193.keytab
--principal=kudu/127.13.149.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.149.212:33805
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790257629041645-13911-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790257629041645-13911-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790257629041645-13911-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 13:48:32.307739 24318 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 13:48:32.307965 24318 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 13:48:32.307999 24318 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 13:48:32.309255 24318 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 13:48:32.309303 24318 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 13:48:32.309341 24318 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.149.193
I20260924 13:48:32.310864 24318 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.149.212:33805
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790257629041645-13911-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790257629041645-13911-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.193.keytab
--principal=kudu/127.13.149.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.149.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790257629041645-13911-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.149.193
--webserver_port=0
--tserver_master_addrs=127.13.149.254:36833
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24318
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.149.193
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790257629041645-13911-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 12:29:37 UTC on ccycloud.quasar-iqzbxn.root.comops.site
build id 4670106
I20260924 13:48:32.311120 24318 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 13:48:32.311336 24318 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 13:48:32.316177 24326 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 13:48:32.316232 24324 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 13:48:32.316352 24323 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 13:48:32.316505 24318 server_base.cc:1034] running on GCE node
I20260924 13:48:32.316712 24318 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 13:48:32.316985 24318 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 13:48:32.318156 24318 hybrid_clock.cc:648] HybridClock initialized: now 1790257712318134 us; error 34 us; skew 500 ppm
Sep 24 13:48:32 dist-test-slave-sjl2 krb5kdc[24211](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257712, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.149.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 13:48:32.320698 24318 init.cc:377] Logged in from keytab as kudu/127.13.149.193@KRBTEST.COM (short username kudu)
I20260924 13:48:32.321204 24318 webserver.cc:460] Webserver started at http://127.13.149.193:34409/ using document root <none> and password file <none>
I20260924 13:48:32.321466 24318 fs_manager.cc:359] Metadata directory not provided
I20260924 13:48:32.321518 24318 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 13:48:32.321647 24318 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 13:48:32.322813 24318 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790257629041645-13911-0/minicluster-data/ts-0/data/instance:
uuid: "df780e365ed44d609e6544e8bffbd5ce"
format_stamp: "Formatted at 2026-09-24 13:48:32 on dist-test-slave-sjl2"
server_key: "befbc1169f0e0732b557acae96049d7c"
server_key_iv: "9c4ed186301aeed5a42f482166cdea59"
server_key_version: "encryptionkey@0"
I20260924 13:48:32.323189 24318 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790257629041645-13911-0/minicluster-data/ts-0/wal/instance:
uuid: "df780e365ed44d609e6544e8bffbd5ce"
format_stamp: "Formatted at 2026-09-24 13:48:32 on dist-test-slave-sjl2"
server_key: "befbc1169f0e0732b557acae96049d7c"
server_key_iv: "9c4ed186301aeed5a42f482166cdea59"
server_key_version: "encryptionkey@0"
I20260924 13:48:32.324486 24318 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260924 13:48:32.325249 24333 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 13:48:32.325407 24318 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260924 13:48:32.325496 24318 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790257629041645-13911-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790257629041645-13911-0/minicluster-data/ts-0/wal
uuid: "df780e365ed44d609e6544e8bffbd5ce"
format_stamp: "Formatted at 2026-09-24 13:48:32 on dist-test-slave-sjl2"
server_key: "befbc1169f0e0732b557acae96049d7c"
server_key_iv: "9c4ed186301aeed5a42f482166cdea59"
server_key_version: "encryptionkey@0"
I20260924 13:48:32.325563 24318 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790257629041645-13911-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790257629041645-13911-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790257629041645-13911-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 13:48:32.339902 24318 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 13:48:32.342070 24318 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 13:48:32.342214 24318 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 13:48:32.342432 24318 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 13:48:32.342728 24318 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 13:48:32.342762 24318 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 13:48:32.342793 24318 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 13:48:32.342816 24318 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 13:48:32.348491 24318 rpc_server.cc:307] RPC server started. Bound to: 127.13.149.193:38533
I20260924 13:48:32.348539 24446 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.149.193:38533 every 8 connection(s)
I20260924 13:48:32.348850 24318 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790257629041645-13911-0/minicluster-data/ts-0/data/info.pb
I20260924 13:48:32.351366 13911 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJN9WRF/build/release/bin/kudu as pid 24318
I20260924 13:48:32.351481 13911 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790257629041645-13911-0/minicluster-data/ts-0/wal/instance
I20260924 13:48:32.351645 13911 external_mini_cluster.cc:1428] Setting key 94d1eb3cb5242d189f7d8684bc2eb756
Sep 24 13:48:32 dist-test-slave-sjl2 krb5kdc[24211](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257712, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.149.193@KRBTEST.COM for kudu/127.13.149.254@KRBTEST.COM
I20260924 13:48:32.359104 24295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:48:32.349558 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.149.193:56959 (local address 127.13.149.254:36833)
0924 13:48:32.349615 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0924 13:48:32.349617 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:48:32.349827 (+ 210us) server_negotiation.cc:407] Connection header received
0924 13:48:32.350338 (+ 511us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:48:32.350340 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:48:32.350352 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:48:32.350403 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:48:32.351510 (+ 1107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:48:32.352131 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:48:32.353611 (+ 1480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:48:32.353918 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:48:32.353999 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:48:32.356118 (+ 2119us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:48:32.356119 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:48:32.356119 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 13:48:32.356127 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:48:32.357599 (+ 1472us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:48:32.358081 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:48:32.358082 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:48:32.358083 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:48:32.358150 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:48:32.358458 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:48:32.358459 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:48:32.358459 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:48:32.358578 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:48:32.358626 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0924 13:48:32.358960 (+ 334us) server_negotiation.cc:299] Negotiation successful
0924 13:48:32.359035 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260924 13:48:32.359427 24449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:48:32.349525 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:36833 (local address 127.13.149.193:56959)
0924 13:48:32.349710 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0924 13:48:32.349747 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0924 13:48:32.350241 (+ 494us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:48:32.350482 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:48:32.350487 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:48:32.351080 (+ 593us) client_negotiation.cc:189] Negotiated authn=SASL
0924 13:48:32.351348 (+ 268us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:48:32.351360 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:48:32.352233 (+ 873us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:48:32.352234 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:48:32.353490 (+ 1256us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:48:32.353494 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:48:32.354009 (+ 515us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:48:32.354010 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:48:32.354123 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:48:32.354658 (+ 535us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 13:48:32.354666 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0924 13:48:32.356023 (+ 1357us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 13:48:32.357687 (+ 1664us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 13:48:32.357696 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 13:48:32.357699 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0924 13:48:32.358007 (+ 308us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 13:48:32.358221 (+ 214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 13:48:32.358222 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 13:48:32.358222 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 13:48:32.358371 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 13:48:32.358635 (+ 264us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 13:48:32.358642 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 13:48:32.358814 (+ 172us) client_negotiation.cc:769] Sending connection context
0924 13:48:32.358931 (+ 117us) client_negotiation.cc:240] Negotiation successful
0924 13:48:32.359106 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":80,"threads_started":1}
WARNING: no policy specified for kudu/127.13.149.194@KRBTEST.COM; defaulting to no policy
I20260924 13:48:32.360028 24447 heartbeater.cc:347] Connected to a master server at 127.13.149.254:36833
I20260924 13:48:32.360122 24447 heartbeater.cc:464] Registering TS with master...
I20260924 13:48:32.360270 24447 heartbeater.cc:511] Master 127.13.149.254:36833 requested a full tablet report, sending...
I20260924 13:48:32.360724 24256 ts_manager.cc:194] Registered new tserver with Master: df780e365ed44d609e6544e8bffbd5ce (127.13.149.193:38533)
I20260924 13:48:32.361344 24256 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.149.193@KRBTEST.COM'} at 127.13.149.193:56959
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.149.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.149.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.194.keytab.
Entry for principal kudu/127.13.149.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.194.keytab.
I20260924 13:48:32.378233 13911 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJN9WRF/build/release/bin/kudu
/tmp/dist-test-taskJN9WRF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790257629041645-13911-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790257629041645-13911-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790257629041645-13911-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790257629041645-13911-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.149.194:0
--local_ip_for_outbound_sockets=127.13.149.194
--webserver_interface=127.13.149.194
--webserver_port=0
--tserver_master_addrs=127.13.149.254:36833
--keytab_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.194.keytab
--principal=kudu/127.13.149.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.149.212:33805
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790257629041645-13911-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790257629041645-13911-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790257629041645-13911-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 13:48:32.457693 24452 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 13:48:32.457917 24452 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 13:48:32.457942 24452 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 13:48:32.459136 24452 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 13:48:32.459182 24452 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 13:48:32.459208 24452 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.149.194
I20260924 13:48:32.460604 24452 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.149.212:33805
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790257629041645-13911-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790257629041645-13911-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.194.keytab
--principal=kudu/127.13.149.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.149.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790257629041645-13911-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.149.194
--webserver_port=0
--tserver_master_addrs=127.13.149.254:36833
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24452
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.149.194
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790257629041645-13911-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 12:29:37 UTC on ccycloud.quasar-iqzbxn.root.comops.site
build id 4670106
I20260924 13:48:32.460803 24452 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 13:48:32.461014 24452 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 13:48:32.465912 24460 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 13:48:32.465940 24458 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 13:48:32.465901 24457 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 13:48:32.466164 24452 server_base.cc:1034] running on GCE node
I20260924 13:48:32.466377 24452 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 13:48:32.466604 24452 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 13:48:32.467762 24452 hybrid_clock.cc:648] HybridClock initialized: now 1790257712467748 us; error 33 us; skew 500 ppm
Sep 24 13:48:32 dist-test-slave-sjl2 krb5kdc[24211](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257712, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.149.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 13:48:32.470407 24452 init.cc:377] Logged in from keytab as kudu/127.13.149.194@KRBTEST.COM (short username kudu)
I20260924 13:48:32.470861 24452 webserver.cc:460] Webserver started at http://127.13.149.194:36129/ using document root <none> and password file <none>
I20260924 13:48:32.471068 24452 fs_manager.cc:359] Metadata directory not provided
I20260924 13:48:32.471113 24452 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 13:48:32.471231 24452 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 13:48:32.472249 24452 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790257629041645-13911-0/minicluster-data/ts-1/data/instance:
uuid: "9fddb53bbb65479ba03bd0aeb30e96d2"
format_stamp: "Formatted at 2026-09-24 13:48:32 on dist-test-slave-sjl2"
server_key: "7d42127d4c3673fce4c9577f2a234c72"
server_key_iv: "912dff196883b96ab569d48eca383141"
server_key_version: "encryptionkey@0"
I20260924 13:48:32.472594 24452 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790257629041645-13911-0/minicluster-data/ts-1/wal/instance:
uuid: "9fddb53bbb65479ba03bd0aeb30e96d2"
format_stamp: "Formatted at 2026-09-24 13:48:32 on dist-test-slave-sjl2"
server_key: "7d42127d4c3673fce4c9577f2a234c72"
server_key_iv: "912dff196883b96ab569d48eca383141"
server_key_version: "encryptionkey@0"
I20260924 13:48:32.473930 24452 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260924 13:48:32.474779 24467 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 13:48:32.474922 24452 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260924 13:48:32.475008 24452 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790257629041645-13911-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790257629041645-13911-0/minicluster-data/ts-1/wal
uuid: "9fddb53bbb65479ba03bd0aeb30e96d2"
format_stamp: "Formatted at 2026-09-24 13:48:32 on dist-test-slave-sjl2"
server_key: "7d42127d4c3673fce4c9577f2a234c72"
server_key_iv: "912dff196883b96ab569d48eca383141"
server_key_version: "encryptionkey@0"
I20260924 13:48:32.475066 24452 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790257629041645-13911-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790257629041645-13911-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790257629041645-13911-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 13:48:32.494267 24452 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 13:48:32.496243 24452 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 13:48:32.496361 24452 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 13:48:32.496557 24452 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 13:48:32.496814 24452 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 13:48:32.496845 24452 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 13:48:32.496865 24452 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 13:48:32.496879 24452 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 13:48:32.502233 24452 rpc_server.cc:307] RPC server started. Bound to: 127.13.149.194:44099
I20260924 13:48:32.502305 24580 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.149.194:44099 every 8 connection(s)
I20260924 13:48:32.502575 24452 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790257629041645-13911-0/minicluster-data/ts-1/data/info.pb
I20260924 13:48:32.502979 13911 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJN9WRF/build/release/bin/kudu as pid 24452
I20260924 13:48:32.503096 13911 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790257629041645-13911-0/minicluster-data/ts-1/wal/instance
I20260924 13:48:32.503300 13911 external_mini_cluster.cc:1428] Setting key 57683857661c59d6cee37d5500096658
Sep 24 13:48:32 dist-test-slave-sjl2 krb5kdc[24211](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257712, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.149.194@KRBTEST.COM for kudu/127.13.149.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.13.149.195@KRBTEST.COM; defaulting to no policy
I20260924 13:48:32.511837 24295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:48:32.503292 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.149.194:33153 (local address 127.13.149.254:36833)
0924 13:48:32.503336 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0924 13:48:32.503338 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:48:32.503592 (+ 254us) server_negotiation.cc:407] Connection header received
0924 13:48:32.504143 (+ 551us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:48:32.504144 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:48:32.504157 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:48:32.504224 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:48:32.505178 (+ 954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:48:32.505668 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:48:32.506625 (+ 957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:48:32.506833 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:48:32.506888 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:48:32.509245 (+ 2357us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:48:32.509247 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:48:32.509247 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 13:48:32.509255 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:48:32.510550 (+ 1295us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:48:32.510919 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:48:32.510920 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:48:32.510920 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:48:32.510980 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:48:32.511193 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:48:32.511194 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:48:32.511194 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:48:32.511319 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:48:32.511376 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0924 13:48:32.511638 (+ 262us) server_negotiation.cc:299] Negotiation successful
0924 13:48:32.511731 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260924 13:48:32.511992 24583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:48:32.503283 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:36833 (local address 127.13.149.194:33153)
0924 13:48:32.503446 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0924 13:48:32.503486 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0924 13:48:32.504055 (+ 569us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:48:32.504282 (+ 227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:48:32.504291 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:48:32.504844 (+ 553us) client_negotiation.cc:189] Negotiated authn=SASL
0924 13:48:32.505092 (+ 248us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:48:32.505098 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:48:32.505750 (+ 652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:48:32.505751 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:48:32.506544 (+ 793us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:48:32.506546 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:48:32.506901 (+ 355us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:48:32.506902 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:48:32.507030 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:48:32.507550 (+ 520us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 13:48:32.507558 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0924 13:48:32.509145 (+ 1587us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 13:48:32.510618 (+ 1473us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 13:48:32.510626 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 13:48:32.510629 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0924 13:48:32.510852 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 13:48:32.511033 (+ 181us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 13:48:32.511034 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 13:48:32.511035 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 13:48:32.511140 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 13:48:32.511372 (+ 232us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 13:48:32.511377 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 13:48:32.511520 (+ 143us) client_negotiation.cc:769] Sending connection context
0924 13:48:32.511609 (+ 89us) client_negotiation.cc:240] Negotiation successful
0924 13:48:32.511767 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":87,"threads_started":1}
I20260924 13:48:32.512473 24581 heartbeater.cc:347] Connected to a master server at 127.13.149.254:36833
I20260924 13:48:32.512571 24581 heartbeater.cc:464] Registering TS with master...
I20260924 13:48:32.512744 24581 heartbeater.cc:511] Master 127.13.149.254:36833 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.149.195@KRBTEST.COM" created.
I20260924 13:48:32.513202 24256 ts_manager.cc:194] Registered new tserver with Master: 9fddb53bbb65479ba03bd0aeb30e96d2 (127.13.149.194:44099)
I20260924 13:48:32.513665 24256 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.149.194@KRBTEST.COM'} at 127.13.149.194:33153
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.149.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.195.keytab.
Entry for principal kudu/127.13.149.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.195.keytab.
I20260924 13:48:32.527798 13911 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJN9WRF/build/release/bin/kudu
/tmp/dist-test-taskJN9WRF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790257629041645-13911-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790257629041645-13911-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790257629041645-13911-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790257629041645-13911-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.149.195:0
--local_ip_for_outbound_sockets=127.13.149.195
--webserver_interface=127.13.149.195
--webserver_port=0
--tserver_master_addrs=127.13.149.254:36833
--keytab_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.195.keytab
--principal=kudu/127.13.149.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.149.212:33805
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790257629041645-13911-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790257629041645-13911-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790257629041645-13911-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 13:48:32.606848 24586 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 13:48:32.607061 24586 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 13:48:32.607084 24586 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 13:48:32.608174 24586 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 13:48:32.608210 24586 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 13:48:32.608235 24586 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.149.195
I20260924 13:48:32.609642 24586 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.149.212:33805
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790257629041645-13911-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790257629041645-13911-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.195.keytab
--principal=kudu/127.13.149.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.149.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790257629041645-13911-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.149.195
--webserver_port=0
--tserver_master_addrs=127.13.149.254:36833
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24586
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.149.195
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790257629041645-13911-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 12:29:37 UTC on ccycloud.quasar-iqzbxn.root.comops.site
build id 4670106
I20260924 13:48:32.609827 24586 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 13:48:32.610004 24586 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 13:48:32.614840 24592 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 13:48:32.614858 24591 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 13:48:32.614850 24594 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 13:48:32.615025 24586 server_base.cc:1034] running on GCE node
I20260924 13:48:32.615255 24586 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 13:48:32.615514 24586 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 13:48:32.616648 24586 hybrid_clock.cc:648] HybridClock initialized: now 1790257712616637 us; error 30 us; skew 500 ppm
Sep 24 13:48:32 dist-test-slave-sjl2 krb5kdc[24211](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257712, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.149.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 13:48:32.618993 24586 init.cc:377] Logged in from keytab as kudu/127.13.149.195@KRBTEST.COM (short username kudu)
I20260924 13:48:32.619390 24586 webserver.cc:460] Webserver started at http://127.13.149.195:37161/ using document root <none> and password file <none>
I20260924 13:48:32.619575 24586 fs_manager.cc:359] Metadata directory not provided
I20260924 13:48:32.619613 24586 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 13:48:32.619707 24586 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 13:48:32.620725 24586 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790257629041645-13911-0/minicluster-data/ts-2/data/instance:
uuid: "9e5399d9c9b24431bf34610d15b7d3dd"
format_stamp: "Formatted at 2026-09-24 13:48:32 on dist-test-slave-sjl2"
server_key: "7f892d638ad3d5a17cdf0959058f90fe"
server_key_iv: "4a5bffabcbefb45a3905e8747e2e16d1"
server_key_version: "encryptionkey@0"
I20260924 13:48:32.621076 24586 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790257629041645-13911-0/minicluster-data/ts-2/wal/instance:
uuid: "9e5399d9c9b24431bf34610d15b7d3dd"
format_stamp: "Formatted at 2026-09-24 13:48:32 on dist-test-slave-sjl2"
server_key: "7f892d638ad3d5a17cdf0959058f90fe"
server_key_iv: "4a5bffabcbefb45a3905e8747e2e16d1"
server_key_version: "encryptionkey@0"
I20260924 13:48:32.622298 24586 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260924 13:48:32.622931 24601 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 13:48:32.623077 24586 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260924 13:48:32.623154 24586 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790257629041645-13911-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790257629041645-13911-0/minicluster-data/ts-2/wal
uuid: "9e5399d9c9b24431bf34610d15b7d3dd"
format_stamp: "Formatted at 2026-09-24 13:48:32 on dist-test-slave-sjl2"
server_key: "7f892d638ad3d5a17cdf0959058f90fe"
server_key_iv: "4a5bffabcbefb45a3905e8747e2e16d1"
server_key_version: "encryptionkey@0"
I20260924 13:48:32.623222 24586 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790257629041645-13911-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790257629041645-13911-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790257629041645-13911-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 13:48:32.642391 24586 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 13:48:32.644492 24586 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 13:48:32.644625 24586 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 13:48:32.644853 24586 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 13:48:32.645183 24586 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 13:48:32.645216 24586 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 13:48:32.645237 24586 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 13:48:32.645248 24586 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 13:48:32.650619 24586 rpc_server.cc:307] RPC server started. Bound to: 127.13.149.195:41767
I20260924 13:48:32.650666 24714 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.149.195:41767 every 8 connection(s)
I20260924 13:48:32.650970 24586 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790257629041645-13911-0/minicluster-data/ts-2/data/info.pb
I20260924 13:48:32.652630 13911 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJN9WRF/build/release/bin/kudu as pid 24586
I20260924 13:48:32.652714 13911 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790257629041645-13911-0/minicluster-data/ts-2/wal/instance
I20260924 13:48:32.652887 13911 external_mini_cluster.cc:1428] Setting key 55a30749a0f9ff8b56f523732fa5bad4
Sep 24 13:48:32 dist-test-slave-sjl2 krb5kdc[24211](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257712, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.149.195@KRBTEST.COM for kudu/127.13.149.254@KRBTEST.COM
I20260924 13:48:32.659693 24295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:48:32.651683 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.149.195:60091 (local address 127.13.149.254:36833)
0924 13:48:32.651763 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0924 13:48:32.651765 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:48:32.651888 (+ 123us) server_negotiation.cc:407] Connection header received
0924 13:48:32.652445 (+ 557us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:48:32.652446 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:48:32.652458 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:48:32.652531 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:48:32.653645 (+ 1114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:48:32.654179 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:48:32.655145 (+ 966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:48:32.655323 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:48:32.655365 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:48:32.657352 (+ 1987us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:48:32.657353 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:48:32.657353 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 13:48:32.657360 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:48:32.658563 (+ 1203us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:48:32.658959 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:48:32.658960 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:48:32.658960 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:48:32.659011 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:48:32.659188 (+ 177us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:48:32.659188 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:48:32.659189 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:48:32.659285 (+ 96us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:48:32.659328 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0924 13:48:32.659571 (+ 243us) server_negotiation.cc:299] Negotiation successful
0924 13:48:32.659627 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260924 13:48:32.659895 24717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:48:32.651610 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:36833 (local address 127.13.149.195:60091)
0924 13:48:32.651762 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0924 13:48:32.651805 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0924 13:48:32.652326 (+ 521us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:48:32.652587 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:48:32.652590 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:48:32.653301 (+ 711us) client_negotiation.cc:189] Negotiated authn=SASL
0924 13:48:32.653568 (+ 267us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:48:32.653573 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:48:32.654250 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:48:32.654251 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:48:32.655081 (+ 830us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:48:32.655083 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:48:32.655401 (+ 318us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:48:32.655402 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:48:32.655503 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:48:32.655951 (+ 448us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 13:48:32.655959 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0924 13:48:32.657272 (+ 1313us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 13:48:32.658639 (+ 1367us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 13:48:32.658647 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 13:48:32.658649 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 13:48:32.658896 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 13:48:32.659063 (+ 167us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 13:48:32.659063 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 13:48:32.659064 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 13:48:32.659153 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 13:48:32.659325 (+ 172us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 13:48:32.659358 (+ 33us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 13:48:32.659474 (+ 116us) client_negotiation.cc:769] Sending connection context
0924 13:48:32.659558 (+ 84us) client_negotiation.cc:240] Negotiation successful
0924 13:48:32.659677 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":60,"threads_started":1}
I20260924 13:48:32.660305 24715 heartbeater.cc:347] Connected to a master server at 127.13.149.254:36833
I20260924 13:48:32.660400 24715 heartbeater.cc:464] Registering TS with master...
I20260924 13:48:32.660574 24715 heartbeater.cc:511] Master 127.13.149.254:36833 requested a full tablet report, sending...
I20260924 13:48:32.660964 24256 ts_manager.cc:194] Registered new tserver with Master: 9e5399d9c9b24431bf34610d15b7d3dd (127.13.149.195:41767)
I20260924 13:48:32.661321 24256 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.149.195@KRBTEST.COM'} at 127.13.149.195:60091
I20260924 13:48:32.664748 13911 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260924 13:48:32.671960 24295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:48:32.666076 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51516 (local address 127.13.149.254:36833)
0924 13:48:32.666129 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0924 13:48:32.666131 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:48:32.666192 (+ 61us) server_negotiation.cc:407] Connection header received
0924 13:48:32.666260 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:48:32.666260 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:48:32.666270 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:48:32.666308 (+ 38us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:48:32.666847 (+ 539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:48:32.667353 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:48:32.668082 (+ 729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:48:32.668269 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:48:32.668386 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:48:32.669706 (+ 1320us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:48:32.669707 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:48:32.669707 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 13:48:32.669713 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:48:32.670864 (+ 1151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:48:32.671236 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:48:32.671237 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:48:32.671237 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:48:32.671292 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:48:32.671438 (+ 146us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:48:32.671439 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:48:32.671439 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:48:32.671582 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:48:32.671662 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0924 13:48:32.671828 (+ 166us) server_negotiation.cc:299] Negotiation successful
0924 13:48:32.671891 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260924 13:48:32.673640 24256 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51516:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 13:48:32.675410 24256 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 13:48:32.681926 24725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:48:32.677586 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.193:38533 (local address 127.0.0.1:41218)
0924 13:48:32.677833 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0924 13:48:32.677844 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0924 13:48:32.677888 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:48:32.678431 (+ 543us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:48:32.678432 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:48:32.678434 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 13:48:32.678482 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:48:32.678484 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:48:32.679522 (+ 1038us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:48:32.679523 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:48:32.680895 (+ 1372us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:48:32.680896 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:48:32.681595 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:48:32.681596 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:48:32.681679 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:48:32.681694 (+ 15us) client_negotiation.cc:769] Sending connection context
0924 13:48:32.681760 (+ 66us) client_negotiation.cc:240] Negotiation successful
0924 13:48:32.681820 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":233,"mutex_wait_us":38,"thread_start_us":69,"threads_started":1}
I20260924 13:48:32.681927 24724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:48:32.677587 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.194:44099 (local address 127.0.0.1:47820)
0924 13:48:32.677970 (+ 383us) negotiation.cc:107] Waiting for socket to connect
0924 13:48:32.677978 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0924 13:48:32.678011 (+ 33us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:48:32.678181 (+ 170us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:48:32.678185 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:48:32.678187 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 13:48:32.678262 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:48:32.678264 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:48:32.679618 (+ 1354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:48:32.679619 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:48:32.680890 (+ 1271us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:48:32.680893 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:48:32.681711 (+ 818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:48:32.681712 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:48:32.681763 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:48:32.681766 (+ 3us) client_negotiation.cc:769] Sending connection context
0924 13:48:32.681808 (+ 42us) client_negotiation.cc:240] Negotiation successful
0924 13:48:32.681840 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":372,"mutex_wait_us":49,"thread_start_us":53,"threads_started":1}
I20260924 13:48:32.682101 24728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:48:32.678249 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51358 (local address 127.13.149.195:41767)
0924 13:48:32.678510 (+ 261us) server_negotiation.cc:206] Beginning negotiation
0924 13:48:32.678515 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 13:48:32.678524 (+ 9us) server_negotiation.cc:407] Connection header received
0924 13:48:32.678558 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:48:32.678560 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:48:32.678575 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:48:32.678651 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 13:48:32.678767 (+ 116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:48:32.679777 (+ 1010us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:48:32.680824 (+ 1047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:48:32.681611 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:48:32.681686 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:48:32.681742 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0924 13:48:32.681978 (+ 236us) server_negotiation.cc:299] Negotiation successful
0924 13:48:32.682019 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":231,"thread_start_us":129,"threads_started":1}
I20260924 13:48:32.682180 24723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:48:32.677583 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.195:41767 (local address 127.0.0.1:51358)
0924 13:48:32.677764 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0924 13:48:32.677806 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0924 13:48:32.677887 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:48:32.678662 (+ 775us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:48:32.678663 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:48:32.678665 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 13:48:32.678701 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:48:32.678702 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:48:32.679864 (+ 1162us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:48:32.679865 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:48:32.680737 (+ 872us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:48:32.680740 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:48:32.681779 (+ 1039us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:48:32.681780 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:48:32.681845 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:48:32.681849 (+ 4us) client_negotiation.cc:769] Sending connection context
0924 13:48:32.681892 (+ 43us) client_negotiation.cc:240] Negotiation successful
0924 13:48:32.681934 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":57,"threads_started":1}
I20260924 13:48:32.682199 24727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:48:32.678076 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41218 (local address 127.13.149.193:38533)
0924 13:48:32.678239 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0924 13:48:32.678243 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 13:48:32.678251 (+ 8us) server_negotiation.cc:407] Connection header received
0924 13:48:32.678304 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:48:32.678306 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:48:32.678320 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:48:32.678552 (+ 232us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 13:48:32.678623 (+ 71us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:48:32.679434 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:48:32.681010 (+ 1576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:48:32.681514 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:48:32.681582 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:48:32.681644 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0924 13:48:32.682069 (+ 425us) server_negotiation.cc:299] Negotiation successful
0924 13:48:32.682120 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127,"thread_start_us":72,"threads_started":1}
I20260924 13:48:32.681968 24726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:48:32.677863 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47820 (local address 127.13.149.194:44099)
0924 13:48:32.678035 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0924 13:48:32.678039 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 13:48:32.678047 (+ 8us) server_negotiation.cc:407] Connection header received
0924 13:48:32.678078 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:48:32.678080 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:48:32.678091 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:48:32.678151 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 13:48:32.678351 (+ 200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:48:32.679519 (+ 1168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:48:32.680989 (+ 1470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:48:32.681651 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:48:32.681713 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:48:32.681755 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0924 13:48:32.681842 (+ 87us) server_negotiation.cc:299] Negotiation successful
0924 13:48:32.681883 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":144,"thread_start_us":94,"threads_started":1}
I20260924 13:48:32.682940 24649 tablet_service.cc:1467] Processing CreateTablet for tablet 2e0850b103034dada7ab8fb9e4d9dd37 (DEFAULT_TABLE table=test-table [id=b68592edb2e649bd835a4b82fe59bdd4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 13:48:32.682947 24381 tablet_service.cc:1467] Processing CreateTablet for tablet 2e0850b103034dada7ab8fb9e4d9dd37 (DEFAULT_TABLE table=test-table [id=b68592edb2e649bd835a4b82fe59bdd4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 13:48:32.683292 24649 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2e0850b103034dada7ab8fb9e4d9dd37. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 13:48:32.683295 24381 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2e0850b103034dada7ab8fb9e4d9dd37. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 13:48:32.683280 24515 tablet_service.cc:1467] Processing CreateTablet for tablet 2e0850b103034dada7ab8fb9e4d9dd37 (DEFAULT_TABLE table=test-table [id=b68592edb2e649bd835a4b82fe59bdd4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 13:48:32.683562 24515 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2e0850b103034dada7ab8fb9e4d9dd37. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 13:48:32.686072 24729 tablet_bootstrap.cc:492] T 2e0850b103034dada7ab8fb9e4d9dd37 P 9e5399d9c9b24431bf34610d15b7d3dd: Bootstrap starting.
I20260924 13:48:32.686544 24730 tablet_bootstrap.cc:492] T 2e0850b103034dada7ab8fb9e4d9dd37 P 9fddb53bbb65479ba03bd0aeb30e96d2: Bootstrap starting.
I20260924 13:48:32.686725 24729 tablet_bootstrap.cc:654] T 2e0850b103034dada7ab8fb9e4d9dd37 P 9e5399d9c9b24431bf34610d15b7d3dd: Neither blocks nor log segments found. Creating new log.
I20260924 13:48:32.686890 24731 tablet_bootstrap.cc:492] T 2e0850b103034dada7ab8fb9e4d9dd37 P df780e365ed44d609e6544e8bffbd5ce: Bootstrap starting.
I20260924 13:48:32.687175 24730 tablet_bootstrap.cc:654] T 2e0850b103034dada7ab8fb9e4d9dd37 P 9fddb53bbb65479ba03bd0aeb30e96d2: Neither blocks nor log segments found. Creating new log.
I20260924 13:48:32.687258 24729 log.cc:824] T 2e0850b103034dada7ab8fb9e4d9dd37 P 9e5399d9c9b24431bf34610d15b7d3dd: Log is configured to *not* fsync() on all Append() calls
I20260924 13:48:32.687425 24731 tablet_bootstrap.cc:654] T 2e0850b103034dada7ab8fb9e4d9dd37 P df780e365ed44d609e6544e8bffbd5ce: Neither blocks nor log segments found. Creating new log.
I20260924 13:48:32.687649 24730 log.cc:824] T 2e0850b103034dada7ab8fb9e4d9dd37 P 9fddb53bbb65479ba03bd0aeb30e96d2: Log is configured to *not* fsync() on all Append() calls
I20260924 13:48:32.688004 24731 log.cc:824] T 2e0850b103034dada7ab8fb9e4d9dd37 P df780e365ed44d609e6544e8bffbd5ce: Log is configured to *not* fsync() on all Append() calls
I20260924 13:48:32.688088 24729 tablet_bootstrap.cc:492] T 2e0850b103034dada7ab8fb9e4d9dd37 P 9e5399d9c9b24431bf34610d15b7d3dd: No bootstrap required, opened a new log
I20260924 13:48:32.688158 24729 ts_tablet_manager.cc:1397] T 2e0850b103034dada7ab8fb9e4d9dd37 P 9e5399d9c9b24431bf34610d15b7d3dd: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260924 13:48:32.688315 24730 tablet_bootstrap.cc:492] T 2e0850b103034dada7ab8fb9e4d9dd37 P 9fddb53bbb65479ba03bd0aeb30e96d2: No bootstrap required, opened a new log
I20260924 13:48:32.688375 24730 ts_tablet_manager.cc:1397] T 2e0850b103034dada7ab8fb9e4d9dd37 P 9fddb53bbb65479ba03bd0aeb30e96d2: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260924 13:48:32.688638 24731 tablet_bootstrap.cc:492] T 2e0850b103034dada7ab8fb9e4d9dd37 P df780e365ed44d609e6544e8bffbd5ce: No bootstrap required, opened a new log
I20260924 13:48:32.688694 24731 ts_tablet_manager.cc:1397] T 2e0850b103034dada7ab8fb9e4d9dd37 P df780e365ed44d609e6544e8bffbd5ce: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260924 13:48:32.690160 24731 raft_consensus.cc:348] T 2e0850b103034dada7ab8fb9e4d9dd37 P df780e365ed44d609e6544e8bffbd5ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df780e365ed44d609e6544e8bffbd5ce" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 38533 } } peers { permanent_uuid: "9fddb53bbb65479ba03bd0aeb30e96d2" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 44099 } } peers { permanent_uuid: "9e5399d9c9b24431bf34610d15b7d3dd" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 41767 } }
I20260924 13:48:32.690294 24731 raft_consensus.cc:374] T 2e0850b103034dada7ab8fb9e4d9dd37 P df780e365ed44d609e6544e8bffbd5ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 13:48:32.690326 24731 raft_consensus.cc:729] T 2e0850b103034dada7ab8fb9e4d9dd37 P df780e365ed44d609e6544e8bffbd5ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: df780e365ed44d609e6544e8bffbd5ce, State: Initialized, Role: FOLLOWER
I20260924 13:48:32.690281 24730 raft_consensus.cc:348] T 2e0850b103034dada7ab8fb9e4d9dd37 P 9fddb53bbb65479ba03bd0aeb30e96d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df780e365ed44d609e6544e8bffbd5ce" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 38533 } } peers { permanent_uuid: "9fddb53bbb65479ba03bd0aeb30e96d2" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 44099 } } peers { permanent_uuid: "9e5399d9c9b24431bf34610d15b7d3dd" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 41767 } }
I20260924 13:48:32.690281 24729 raft_consensus.cc:348] T 2e0850b103034dada7ab8fb9e4d9dd37 P 9e5399d9c9b24431bf34610d15b7d3dd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df780e365ed44d609e6544e8bffbd5ce" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 38533 } } peers { permanent_uuid: "9fddb53bbb65479ba03bd0aeb30e96d2" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 44099 } } peers { permanent_uuid: "9e5399d9c9b24431bf34610d15b7d3dd" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 41767 } }
I20260924 13:48:32.690431 24730 raft_consensus.cc:374] T 2e0850b103034dada7ab8fb9e4d9dd37 P 9fddb53bbb65479ba03bd0aeb30e96d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 13:48:32.690434 24729 raft_consensus.cc:374] T 2e0850b103034dada7ab8fb9e4d9dd37 P 9e5399d9c9b24431bf34610d15b7d3dd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 13:48:32.690441 24731 consensus_queue.cc:260] T 2e0850b103034dada7ab8fb9e4d9dd37 P df780e365ed44d609e6544e8bffbd5ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df780e365ed44d609e6544e8bffbd5ce" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 38533 } } peers { permanent_uuid: "9fddb53bbb65479ba03bd0aeb30e96d2" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 44099 } } peers { permanent_uuid: "9e5399d9c9b24431bf34610d15b7d3dd" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 41767 } }
I20260924 13:48:32.690474 24729 raft_consensus.cc:729] T 2e0850b103034dada7ab8fb9e4d9dd37 P 9e5399d9c9b24431bf34610d15b7d3dd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e5399d9c9b24431bf34610d15b7d3dd, State: Initialized, Role: FOLLOWER
I20260924 13:48:32.690474 24730 raft_consensus.cc:729] T 2e0850b103034dada7ab8fb9e4d9dd37 P 9fddb53bbb65479ba03bd0aeb30e96d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9fddb53bbb65479ba03bd0aeb30e96d2, State: Initialized, Role: FOLLOWER
I20260924 13:48:32.690608 24729 consensus_queue.cc:260] T 2e0850b103034dada7ab8fb9e4d9dd37 P 9e5399d9c9b24431bf34610d15b7d3dd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df780e365ed44d609e6544e8bffbd5ce" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 38533 } } peers { permanent_uuid: "9fddb53bbb65479ba03bd0aeb30e96d2" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 44099 } } peers { permanent_uuid: "9e5399d9c9b24431bf34610d15b7d3dd" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 41767 } }
I20260924 13:48:32.690609 24730 consensus_queue.cc:260] T 2e0850b103034dada7ab8fb9e4d9dd37 P 9fddb53bbb65479ba03bd0aeb30e96d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df780e365ed44d609e6544e8bffbd5ce" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 38533 } } peers { permanent_uuid: "9fddb53bbb65479ba03bd0aeb30e96d2" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 44099 } } peers { permanent_uuid: "9e5399d9c9b24431bf34610d15b7d3dd" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 41767 } }
I20260924 13:48:32.690716 24731 ts_tablet_manager.cc:1428] T 2e0850b103034dada7ab8fb9e4d9dd37 P df780e365ed44d609e6544e8bffbd5ce: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260924 13:48:32.690848 24447 heartbeater.cc:503] Master 127.13.149.254:36833 was elected leader, sending a full tablet report...
I20260924 13:48:32.690869 24729 ts_tablet_manager.cc:1428] T 2e0850b103034dada7ab8fb9e4d9dd37 P 9e5399d9c9b24431bf34610d15b7d3dd: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260924 13:48:32.690869 24730 ts_tablet_manager.cc:1428] T 2e0850b103034dada7ab8fb9e4d9dd37 P 9fddb53bbb65479ba03bd0aeb30e96d2: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260924 13:48:32.690971 24581 heartbeater.cc:503] Master 127.13.149.254:36833 was elected leader, sending a full tablet report...
I20260924 13:48:32.691074 24715 heartbeater.cc:503] Master 127.13.149.254:36833 was elected leader, sending a full tablet report...
W20260924 13:48:32.753808 24582 tablet.cc:2369] T 2e0850b103034dada7ab8fb9e4d9dd37 P 9fddb53bbb65479ba03bd0aeb30e96d2: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260924 13:48:32.849658 24448 tablet.cc:2369] T 2e0850b103034dada7ab8fb9e4d9dd37 P df780e365ed44d609e6544e8bffbd5ce: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260924 13:48:32.901551 24716 tablet.cc:2369] T 2e0850b103034dada7ab8fb9e4d9dd37 P 9e5399d9c9b24431bf34610d15b7d3dd: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260924 13:48:32.930701 24735 raft_consensus.cc:482] T 2e0850b103034dada7ab8fb9e4d9dd37 P df780e365ed44d609e6544e8bffbd5ce [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 13:48:32.930824 24735 raft_consensus.cc:504] T 2e0850b103034dada7ab8fb9e4d9dd37 P df780e365ed44d609e6544e8bffbd5ce [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df780e365ed44d609e6544e8bffbd5ce" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 38533 } } peers { permanent_uuid: "9fddb53bbb65479ba03bd0aeb30e96d2" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 44099 } } peers { permanent_uuid: "9e5399d9c9b24431bf34610d15b7d3dd" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 41767 } }
I20260924 13:48:32.931185 24735 leader_election.cc:288] T 2e0850b103034dada7ab8fb9e4d9dd37 P df780e365ed44d609e6544e8bffbd5ce [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9fddb53bbb65479ba03bd0aeb30e96d2 (127.13.149.194:44099), 9e5399d9c9b24431bf34610d15b7d3dd (127.13.149.195:41767)
I20260924 13:48:32.934005 24728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:48:32.931420 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.149.193:50845 (local address 127.13.149.195:41767)
0924 13:48:32.931469 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0924 13:48:32.931471 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:48:32.931562 (+ 91us) server_negotiation.cc:407] Connection header received
0924 13:48:32.931654 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:48:32.931655 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:48:32.931664 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:48:32.931709 (+ 45us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 13:48:32.931859 (+ 150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:48:32.932316 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:48:32.933322 (+ 1006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:48:32.933691 (+ 369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:48:32.933754 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:48:32.933763 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0924 13:48:32.933901 (+ 138us) server_negotiation.cc:299] Negotiation successful
0924 13:48:32.933939 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260924 13:48:32.934027 24726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:48:32.931510 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.149.193:55573 (local address 127.13.149.194:44099)
0924 13:48:32.931579 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0924 13:48:32.931581 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:48:32.931608 (+ 27us) server_negotiation.cc:407] Connection header received
0924 13:48:32.931635 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:48:32.931636 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:48:32.931643 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:48:32.931684 (+ 41us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 13:48:32.931842 (+ 158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:48:32.932322 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:48:32.933344 (+ 1022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:48:32.933719 (+ 375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:48:32.933781 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:48:32.933789 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0924 13:48:32.933927 (+ 138us) server_negotiation.cc:299] Negotiation successful
0924 13:48:32.933961 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260924 13:48:32.934049 24739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:48:32.931365 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.194:44099 (local address 127.13.149.193:55573)
0924 13:48:32.931514 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0924 13:48:32.931522 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0924 13:48:32.931575 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:48:32.931694 (+ 119us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:48:32.931695 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:48:32.931696 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 13:48:32.931769 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:48:32.931772 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:48:32.932394 (+ 622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:48:32.932395 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:48:32.933263 (+ 868us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:48:32.933265 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:48:32.933791 (+ 526us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:48:32.933792 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:48:32.933859 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:48:32.933862 (+ 3us) client_negotiation.cc:769] Sending connection context
0924 13:48:32.933905 (+ 43us) client_negotiation.cc:240] Negotiation successful
0924 13:48:32.933946 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"mutex_wait_us":13,"thread_start_us":81,"threads_started":1}
I20260924 13:48:32.934056 24738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:48:32.931354 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.195:41767 (local address 127.13.149.193:50845)
0924 13:48:32.931490 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0924 13:48:32.931502 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0924 13:48:32.931589 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:48:32.931723 (+ 134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:48:32.931724 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:48:32.931725 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 13:48:32.931800 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:48:32.931802 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:48:32.932400 (+ 598us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:48:32.932401 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:48:32.933237 (+ 836us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:48:32.933239 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:48:32.933770 (+ 531us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:48:32.933771 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:48:32.933835 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:48:32.933839 (+ 4us) client_negotiation.cc:769] Sending connection context
0924 13:48:32.933882 (+ 43us) client_negotiation.cc:240] Negotiation successful
0924 13:48:32.933930 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"mutex_wait_us":3,"thread_start_us":65,"threads_started":1}
I20260924 13:48:32.934392 24535 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2e0850b103034dada7ab8fb9e4d9dd37" candidate_uuid: "df780e365ed44d609e6544e8bffbd5ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9fddb53bbb65479ba03bd0aeb30e96d2" is_pre_election: true
I20260924 13:48:32.934573 24535 raft_consensus.cc:2454] T 2e0850b103034dada7ab8fb9e4d9dd37 P 9fddb53bbb65479ba03bd0aeb30e96d2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate df780e365ed44d609e6544e8bffbd5ce in term 0.
I20260924 13:48:32.934547 24669 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2e0850b103034dada7ab8fb9e4d9dd37" candidate_uuid: "df780e365ed44d609e6544e8bffbd5ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e5399d9c9b24431bf34610d15b7d3dd" is_pre_election: true
I20260924 13:48:32.934693 24669 raft_consensus.cc:2454] T 2e0850b103034dada7ab8fb9e4d9dd37 P 9e5399d9c9b24431bf34610d15b7d3dd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate df780e365ed44d609e6544e8bffbd5ce in term 0.
I20260924 13:48:32.934808 24335 leader_election.cc:302] T 2e0850b103034dada7ab8fb9e4d9dd37 P df780e365ed44d609e6544e8bffbd5ce [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9fddb53bbb65479ba03bd0aeb30e96d2, df780e365ed44d609e6544e8bffbd5ce; no voters:
I20260924 13:48:32.934937 24735 raft_consensus.cc:2781] T 2e0850b103034dada7ab8fb9e4d9dd37 P df780e365ed44d609e6544e8bffbd5ce [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 13:48:32.934991 24735 raft_consensus.cc:482] T 2e0850b103034dada7ab8fb9e4d9dd37 P df780e365ed44d609e6544e8bffbd5ce [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 13:48:32.935019 24735 raft_consensus.cc:3037] T 2e0850b103034dada7ab8fb9e4d9dd37 P df780e365ed44d609e6544e8bffbd5ce [term 0 FOLLOWER]: Advancing to term 1
I20260924 13:48:32.935684 24735 raft_consensus.cc:504] T 2e0850b103034dada7ab8fb9e4d9dd37 P df780e365ed44d609e6544e8bffbd5ce [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df780e365ed44d609e6544e8bffbd5ce" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 38533 } } peers { permanent_uuid: "9fddb53bbb65479ba03bd0aeb30e96d2" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 44099 } } peers { permanent_uuid: "9e5399d9c9b24431bf34610d15b7d3dd" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 41767 } }
I20260924 13:48:32.935808 24735 leader_election.cc:288] T 2e0850b103034dada7ab8fb9e4d9dd37 P df780e365ed44d609e6544e8bffbd5ce [CANDIDATE]: Term 1 election: Requested vote from peers 9fddb53bbb65479ba03bd0aeb30e96d2 (127.13.149.194:44099), 9e5399d9c9b24431bf34610d15b7d3dd (127.13.149.195:41767)
I20260924 13:48:32.935983 24535 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2e0850b103034dada7ab8fb9e4d9dd37" candidate_uuid: "df780e365ed44d609e6544e8bffbd5ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9fddb53bbb65479ba03bd0aeb30e96d2"
I20260924 13:48:32.936014 24669 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2e0850b103034dada7ab8fb9e4d9dd37" candidate_uuid: "df780e365ed44d609e6544e8bffbd5ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e5399d9c9b24431bf34610d15b7d3dd"
I20260924 13:48:32.936064 24535 raft_consensus.cc:3037] T 2e0850b103034dada7ab8fb9e4d9dd37 P 9fddb53bbb65479ba03bd0aeb30e96d2 [term 0 FOLLOWER]: Advancing to term 1
I20260924 13:48:32.936090 24669 raft_consensus.cc:3037] T 2e0850b103034dada7ab8fb9e4d9dd37 P 9e5399d9c9b24431bf34610d15b7d3dd [term 0 FOLLOWER]: Advancing to term 1
I20260924 13:48:32.936787 24535 raft_consensus.cc:2454] T 2e0850b103034dada7ab8fb9e4d9dd37 P 9fddb53bbb65479ba03bd0aeb30e96d2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate df780e365ed44d609e6544e8bffbd5ce in term 1.
I20260924 13:48:32.936811 24669 raft_consensus.cc:2454] T 2e0850b103034dada7ab8fb9e4d9dd37 P 9e5399d9c9b24431bf34610d15b7d3dd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate df780e365ed44d609e6544e8bffbd5ce in term 1.
I20260924 13:48:32.936975 24336 leader_election.cc:302] T 2e0850b103034dada7ab8fb9e4d9dd37 P df780e365ed44d609e6544e8bffbd5ce [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e5399d9c9b24431bf34610d15b7d3dd, df780e365ed44d609e6544e8bffbd5ce; no voters:
I20260924 13:48:32.937081 24735 raft_consensus.cc:2781] T 2e0850b103034dada7ab8fb9e4d9dd37 P df780e365ed44d609e6544e8bffbd5ce [term 1 FOLLOWER]: Leader election won for term 1
I20260924 13:48:32.937243 24735 raft_consensus.cc:686] T 2e0850b103034dada7ab8fb9e4d9dd37 P df780e365ed44d609e6544e8bffbd5ce [term 1 LEADER]: Becoming Leader. State: Replica: df780e365ed44d609e6544e8bffbd5ce, State: Running, Role: LEADER
I20260924 13:48:32.937330 24735 consensus_queue.cc:237] T 2e0850b103034dada7ab8fb9e4d9dd37 P df780e365ed44d609e6544e8bffbd5ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df780e365ed44d609e6544e8bffbd5ce" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 38533 } } peers { permanent_uuid: "9fddb53bbb65479ba03bd0aeb30e96d2" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 44099 } } peers { permanent_uuid: "9e5399d9c9b24431bf34610d15b7d3dd" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 41767 } }
I20260924 13:48:32.938158 24255 catalog_manager.cc:5495] T 2e0850b103034dada7ab8fb9e4d9dd37 P df780e365ed44d609e6544e8bffbd5ce reported cstate change: term changed from 0 to 1, leader changed from <none> to df780e365ed44d609e6544e8bffbd5ce (127.13.149.193). New cstate: current_term: 1 leader_uuid: "df780e365ed44d609e6544e8bffbd5ce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df780e365ed44d609e6544e8bffbd5ce" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 38533 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9fddb53bbb65479ba03bd0aeb30e96d2" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 44099 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e5399d9c9b24431bf34610d15b7d3dd" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 41767 } health_report { overall_health: UNKNOWN } } }
I20260924 13:48:32.964270 24727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:48:32.962030 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41226 (local address 127.13.149.193:38533)
0924 13:48:32.962097 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0924 13:48:32.962099 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:48:32.962107 (+ 8us) server_negotiation.cc:407] Connection header received
0924 13:48:32.962145 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:48:32.962146 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:48:32.962163 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:48:32.962220 (+ 57us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 13:48:32.962344 (+ 124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:48:32.962817 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:48:32.963586 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:48:32.963783 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:48:32.963836 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:48:32.963974 (+ 138us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 13:48:32.964081 (+ 107us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 13:48:32.964150 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0924 13:48:32.964170 (+ 20us) server_negotiation.cc:299] Negotiation successful
0924 13:48:32.964191 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260924 13:48:32.966133 24535 raft_consensus.cc:1261] T 2e0850b103034dada7ab8fb9e4d9dd37 P 9fddb53bbb65479ba03bd0aeb30e96d2 [term 1 FOLLOWER]: Refusing update from remote peer df780e365ed44d609e6544e8bffbd5ce: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 13:48:32.966133 24669 raft_consensus.cc:1261] T 2e0850b103034dada7ab8fb9e4d9dd37 P 9e5399d9c9b24431bf34610d15b7d3dd [term 1 FOLLOWER]: Refusing update from remote peer df780e365ed44d609e6544e8bffbd5ce: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 13:48:32.966465 24740 consensus_queue.cc:1035] T 2e0850b103034dada7ab8fb9e4d9dd37 P df780e365ed44d609e6544e8bffbd5ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e5399d9c9b24431bf34610d15b7d3dd" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 41767 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 13:48:32.966586 24735 consensus_queue.cc:1035] T 2e0850b103034dada7ab8fb9e4d9dd37 P df780e365ed44d609e6544e8bffbd5ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "9fddb53bbb65479ba03bd0aeb30e96d2" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 44099 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 13:48:32.968657 24743 mvcc.cc:205] Tried to move back new op lower bound from 7332895592306892800 to 7332895592191897600. Current Snapshot: MvccSnapshot[applied={T|T < 7332895592306892800}]
I20260924 13:48:32.971350 24256 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51516:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 13:48:32.971463 24256 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51516:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 13:48:32.972231 24256 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1dd3595bceea42e385bf25a41f15e80a: Sending DeleteTablet for 3 replicas of tablet 2e0850b103034dada7ab8fb9e4d9dd37
I20260924 13:48:32.972600 24649 tablet_service.cc:1514] Processing DeleteTablet for tablet 2e0850b103034dada7ab8fb9e4d9dd37 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 13:48:32 UTC) from {username='kudu', principal='kudu/127.13.149.254@KRBTEST.COM'} at 127.0.0.1:51358
I20260924 13:48:32.972666 24381 tablet_service.cc:1514] Processing DeleteTablet for tablet 2e0850b103034dada7ab8fb9e4d9dd37 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 13:48:32 UTC) from {username='kudu', principal='kudu/127.13.149.254@KRBTEST.COM'} at 127.0.0.1:41218
I20260924 13:48:32.972882 13911 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJN9WRF/build/release/bin/kudu with pid 24318
I20260924 13:48:32.972946 24515 tablet_service.cc:1514] Processing DeleteTablet for tablet 2e0850b103034dada7ab8fb9e4d9dd37 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 13:48:32 UTC) from {username='kudu', principal='kudu/127.13.149.254@KRBTEST.COM'} at 127.0.0.1:47820
I20260924 13:48:32.973627 24753 tablet_replica.cc:330] T 2e0850b103034dada7ab8fb9e4d9dd37 P 9e5399d9c9b24431bf34610d15b7d3dd: stopping tablet replica
I20260924 13:48:32.973726 24753 raft_consensus.cc:2229] T 2e0850b103034dada7ab8fb9e4d9dd37 P 9e5399d9c9b24431bf34610d15b7d3dd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 13:48:32.973830 24753 raft_consensus.cc:2258] T 2e0850b103034dada7ab8fb9e4d9dd37 P 9e5399d9c9b24431bf34610d15b7d3dd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 13:48:32.973837 24755 tablet_replica.cc:330] T 2e0850b103034dada7ab8fb9e4d9dd37 P 9fddb53bbb65479ba03bd0aeb30e96d2: stopping tablet replica
I20260924 13:48:32.973953 24755 raft_consensus.cc:2229] T 2e0850b103034dada7ab8fb9e4d9dd37 P 9fddb53bbb65479ba03bd0aeb30e96d2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 13:48:32.974046 24755 raft_consensus.cc:2258] T 2e0850b103034dada7ab8fb9e4d9dd37 P 9fddb53bbb65479ba03bd0aeb30e96d2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 13:48:32.974476 24755 ts_tablet_manager.cc:1905] T 2e0850b103034dada7ab8fb9e4d9dd37 P 9fddb53bbb65479ba03bd0aeb30e96d2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 13:48:32.974797 24753 ts_tablet_manager.cc:1905] T 2e0850b103034dada7ab8fb9e4d9dd37 P 9e5399d9c9b24431bf34610d15b7d3dd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 13:48:32.976540 24753 ts_tablet_manager.cc:1918] T 2e0850b103034dada7ab8fb9e4d9dd37 P 9e5399d9c9b24431bf34610d15b7d3dd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 13:48:32.976545 24755 ts_tablet_manager.cc:1918] T 2e0850b103034dada7ab8fb9e4d9dd37 P 9fddb53bbb65479ba03bd0aeb30e96d2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 13:48:32.976619 24755 log.cc:1196] T 2e0850b103034dada7ab8fb9e4d9dd37 P 9fddb53bbb65479ba03bd0aeb30e96d2: Deleting WAL directory at /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790257629041645-13911-0/minicluster-data/ts-1/wal/wals/2e0850b103034dada7ab8fb9e4d9dd37
I20260924 13:48:32.976629 24753 log.cc:1196] T 2e0850b103034dada7ab8fb9e4d9dd37 P 9e5399d9c9b24431bf34610d15b7d3dd: Deleting WAL directory at /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790257629041645-13911-0/minicluster-data/ts-2/wal/wals/2e0850b103034dada7ab8fb9e4d9dd37
I20260924 13:48:32.976841 24755 ts_tablet_manager.cc:1939] T 2e0850b103034dada7ab8fb9e4d9dd37 P 9fddb53bbb65479ba03bd0aeb30e96d2: Deleting consensus metadata
I20260924 13:48:32.976982 24753 ts_tablet_manager.cc:1939] T 2e0850b103034dada7ab8fb9e4d9dd37 P 9e5399d9c9b24431bf34610d15b7d3dd: Deleting consensus metadata
I20260924 13:48:32.977384 24242 catalog_manager.cc:4841] TS 9e5399d9c9b24431bf34610d15b7d3dd (127.13.149.195:41767): tablet 2e0850b103034dada7ab8fb9e4d9dd37 (table test-table [id=b68592edb2e649bd835a4b82fe59bdd4]) successfully deleted
I20260924 13:48:32.977385 24241 catalog_manager.cc:4841] TS 9fddb53bbb65479ba03bd0aeb30e96d2 (127.13.149.194:44099): tablet 2e0850b103034dada7ab8fb9e4d9dd37 (table test-table [id=b68592edb2e649bd835a4b82fe59bdd4]) successfully deleted
W20260924 13:48:32.979671 24243 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.13.149.193:38533 (error 108)
I20260924 13:48:32.979877 13911 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJN9WRF/build/release/bin/kudu with pid 24452
I20260924 13:48:32.980458 24724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:48:32.980016 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.193:38533 (local address 127.0.0.1:41228)
0924 13:48:32.980377 (+ 361us) negotiation.cc:107] Waiting for socket to connect
0924 13:48:32.980421 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.149.193:38533: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":339}
W20260924 13:48:32.980595 24243 catalog_manager.cc:4584] TS df780e365ed44d609e6544e8bffbd5ce (127.13.149.193:38533): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 2e0850b103034dada7ab8fb9e4d9dd37: Network error: Client connection negotiation failed: client connection to 127.13.149.193:38533: connect: Connection refused (error 111)
I20260924 13:48:32.984144 13911 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJN9WRF/build/release/bin/kudu with pid 24586
I20260924 13:48:32.988397 13911 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJN9WRF/build/release/bin/kudu with pid 24224
2026-09-24T13:48:32Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (2013 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790257629041645-13911-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 13:48:33 dist-test-slave-sjl2 krb5kdc[24759](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 13:48:33 dist-test-slave-sjl2 krb5kdc[24759](info): set up 2 sockets
Sep 24 13:48:33 dist-test-slave-sjl2 krb5kdc[24759](info): commencing operation
krb5kdc: starting...
W20260924 13:48:33.954541 13911 mini_kdc.cc:121] Time spent starting KDC: real 0.947s user 0.001s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790257629041645-13911-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790257629041645-13911-0/krb5kdc/test-user.keytab.
Sep 24 13:48:34 dist-test-slave-sjl2 krb5kdc[24759](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257714, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T13:48:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T13:48:34Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.13.149.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.13.149.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.13.149.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790257629041645-13911-0/krb5kdc/rangeradmin_127.13.149.212@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.13.149.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790257629041645-13911-0/krb5kdc/rangeradmin_127.13.149.212@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.13.149.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.13.149.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.13.149.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790257629041645-13911-0/krb5kdc/rangerlookup_127.13.149.212@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.13.149.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790257629041645-13911-0/krb5kdc/rangerlookup_127.13.149.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.13.149.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.13.149.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.13.149.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790257629041645-13911-0/krb5kdc/HTTP_127.13.149.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.13.149.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790257629041645-13911-0/krb5kdc/HTTP_127.13.149.212@KRBTEST.COM.keytab.
I20260924 13:48:34.125528 13911 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790257629041645-13911-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskJN9WRF/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790257629041645-13911-0/minicluster-data/postgres -l logfile start
2026-09-24 13:48:36.650 UTC [24789] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-09-24 13:48:36.650 UTC [24789] LOG: listening on IPv4 address "127.13.149.212", port 34565
2026-09-24 13:48:36.654 UTC [24789] LOG: listening on Unix socket "/tmp/.s.PGSQL.34565"
2026-09-24 13:48:36.682 UTC [24792] LOG: database system was shut down at 2026-09-24 13:48:35 UTC
2026-09-24 13:48:36.685 UTC [24789] LOG: database system is ready to accept connections
I20260924 13:48:37.550097 13911 mini_postgres.cc:96] Postgres bound to 34565
2026-09-24 13:48:37.555 UTC [24800] FATAL: database "slave" does not exist
127.13.149.212:34565 - accepting connections
I20260924 13:48:37.556445 13911 mini_ranger.cc:161] Starting Ranger...
I20260924 13:48:37.571126 13911 mini_ranger.cc:84] Created miniranger Postgres user
I20260924 13:48:38.066262 13911 mini_ranger.cc:87] Created ranger Postgres database
I20260924 13:48:38.066329 13911 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790257629041645-13911-0/minicluster-data/ranger-admin
2026-09-24 13:48:38,334 [I] DB FLAVOR :POSTGRES
2026-09-24 13:48:38,335 [I] --------- Verifying Ranger DB connection ---------
2026-09-24 13:48:38,335 [I] Checking connection..
2026-09-24 13:48:38,335 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJN9WRF/build/release/bin/postgresql.jar:/tmp/dist-test-taskJN9WRF/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.149.212:34565/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-09-24 13:48:38,599 [I] Checking connection passed.
2026-09-24 13:48:38,599 [I] --------- Verifying version history table ---------
2026-09-24 13:48:38,599 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJN9WRF/build/release/bin/postgresql.jar:/tmp/dist-test-taskJN9WRF/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.149.212:34565/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-24 13:48:38,836 [I] Table x_db_version_h does not exist in database ranger
2026-09-24 13:48:38,836 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJN9WRF/build/release/bin/postgresql.jar:/tmp/dist-test-taskJN9WRF/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.149.212:34565/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-24 13:48:39,099 [I] Table x_db_version_h does not exist in database ranger
2026-09-24 13:48:39,099 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-09-24 13:48:39,099 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJN9WRF/build/release/bin/postgresql.jar:/tmp/dist-test-taskJN9WRF/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.149.212:34565/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskJN9WRF/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-09-24 13:48:39.358 UTC [24864] WARNING: there is no transaction in progress
2026-09-24 13:48:39,363 [I] create_dbversion_catalog.sql file imported successfully
2026-09-24 13:48:39,364 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJN9WRF/build/release/bin/postgresql.jar:/tmp/dist-test-taskJN9WRF/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.149.212:34565/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-24 13:48:39,629 [I] Table x_db_version_h already exists in database 'ranger'
2026-09-24 13:48:39,629 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-24 13:48:39,629 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJN9WRF/build/release/bin/postgresql.jar:/tmp/dist-test-taskJN9WRF/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.149.212:34565/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-09-24 13:48:39,876 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJN9WRF/build/release/bin/postgresql.jar:/tmp/dist-test-taskJN9WRF/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.149.212:34565/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-09-24 13:48:40,136 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJN9WRF/build/release/bin/postgresql.jar:/tmp/dist-test-taskJN9WRF/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.149.212:34565/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-sjl2.c.gcp-upstream.internal','N') ;"
2026-09-24 13:48:40,357 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJN9WRF/build/release/bin/postgresql.jar:/tmp/dist-test-taskJN9WRF/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.149.212:34565/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-24 13:48:40,618 [I] Table x_portal_user does not exist in database ranger
2026-09-24 13:48:40,618 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJN9WRF/build/release/bin/postgresql.jar:/tmp/dist-test-taskJN9WRF/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.149.212:34565/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-24 13:48:40,878 [I] Table x_policy_ref_group does not exist in database ranger
2026-09-24 13:48:40,878 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJN9WRF/build/release/bin/postgresql.jar:/tmp/dist-test-taskJN9WRF/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.149.212:34565/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-24 13:48:41,145 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJN9WRF/build/release/bin/postgresql.jar:/tmp/dist-test-taskJN9WRF/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.149.212:34565/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-24 13:48:41,403 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-09-24 13:48:41,403 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJN9WRF/build/release/bin/postgresql.jar:/tmp/dist-test-taskJN9WRF/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.149.212:34565/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskJN9WRF/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-09-24 13:48:42.371 UTC [24972] WARNING: there is no transaction in progress
2026-09-24 13:48:42.387 UTC [24972] WARNING: there is no transaction in progress
2026-09-24 13:48:42.404 UTC [24972] WARNING: there is no transaction in progress
2026-09-24 13:48:42.419 UTC [24972] WARNING: there is no transaction in progress
2026-09-24 13:48:42.435 UTC [24972] WARNING: there is no transaction in progress
2026-09-24 13:48:42.454 UTC [24972] WARNING: there is no transaction in progress
2026-09-24 13:48:42.518 UTC [24972] WARNING: there is no transaction in progress
2026-09-24 13:48:42.526 UTC [24972] WARNING: there is no transaction in progress
2026-09-24 13:48:42.541 UTC [24972] WARNING: there is no transaction in progress
2026-09-24 13:48:42.552 UTC [24972] WARNING: there is no transaction in progress
2026-09-24 13:48:42.563 UTC [24972] WARNING: there is no transaction in progress
2026-09-24 13:48:42.577 UTC [24972] WARNING: there is no transaction in progress
2026-09-24 13:48:42.588 UTC [24972] WARNING: there is no transaction in progress
2026-09-24 13:48:42.594 UTC [24972] WARNING: there is no transaction in progress
2026-09-24 13:48:42.602 UTC [24972] WARNING: there is no transaction in progress
2026-09-24 13:48:42.610 UTC [24972] WARNING: there is no transaction in progress
2026-09-24 13:48:43,304 [I] ranger_core_db_postgres.sql file imported successfully
2026-09-24 13:48:43,304 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJN9WRF/build/release/bin/postgresql.jar:/tmp/dist-test-taskJN9WRF/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.149.212:34565/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-09-24 13:48:43,546 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-09-24 13:48:43,547 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJN9WRF/build/release/bin/postgresql.jar:/tmp/dist-test-taskJN9WRF/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.149.212:34565/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-24 13:48:43,792 [I] Table x_portal_user already exists in database 'ranger'
2026-09-24 13:48:43,792 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJN9WRF/build/release/bin/postgresql.jar:/tmp/dist-test-taskJN9WRF/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.149.212:34565/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-24 13:48:44,048 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-09-24 13:48:44,048 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJN9WRF/build/release/bin/postgresql.jar:/tmp/dist-test-taskJN9WRF/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.149.212:34565/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-24 13:48:44,308 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJN9WRF/build/release/bin/postgresql.jar:/tmp/dist-test-taskJN9WRF/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.149.212:34565/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-24 13:48:44,566 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJN9WRF/build/release/bin/postgresql.jar:/tmp/dist-test-taskJN9WRF/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.149.212:34565/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-sjl2.c.gcp-upstream.internal';"
2026-09-24 13:48:44,815 [I] CORE_DB_SCHEMA import status has been updated
2026-09-24 13:48:44,815 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJN9WRF/build/release/bin/postgresql.jar:/tmp/dist-test-taskJN9WRF/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.149.212:34565/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-09-24 13:48:45,048 [I] DB_PATCHES have already been applied
I20260924 13:48:45.054191 13911 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790257629041645-13911-0/minicluster-data/ranger-admin:/tmp/dist-test-taskJN9WRF/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskJN9WRF/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskJN9WRF/build/release/bin/postgresql.jar:/tmp/dist-test-taskJN9WRF/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260924 13:48:45.054261 13911 mini_ranger.cc:193] Using host: 127.13.149.212
I20260924 13:48:45.057317 13911 mini_ranger.cc:227] Ranger admin URL: http://127.13.149.212:55809
Sep 24, 2026 1:48:45 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 24, 2026 1:48:45 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 24, 2026 1:48:45 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790257629041645-13911-0/minicluster-data/ranger-admin/ews/webapp
Sep 24, 2026 1:48:45 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790257629041645-13911-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Sep 24, 2026 1:48:45 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790257629041645-13911-0/minicluster-data/ranger-admin/ews/webapp] .....
Sep 24, 2026 1:48:45 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790257629041645-13911-0/minicluster-data/ranger-admin/ews/webapp].
Sep 24, 2026 1:48:45 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260924 13:48:46.076679 13911 mini_ranger.cc:160] Time spent starting Ranger: real 8.520s user 0.000s sys 0.008s
Sep 24, 2026 1:49:05 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Sep 24, 2026 1:49:05 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.UserREST
Sep 24, 2026 1:49:05 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Sep 24, 2026 1:49:05 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Sep 24, 2026 1:49:05 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Sep 24, 2026 1:49:05 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Sep 24, 2026 1:49:05 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Sep 24, 2026 1:49:05 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Sep 24, 2026 1:49:05 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Sep 24, 2026 1:49:05 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Sep 24, 2026 1:49:05 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Sep 24, 2026 1:49:05 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Sep 24, 2026 1:49:05 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Sep 24, 2026 1:49:05 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Sep 24, 2026 1:49:05 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Sep 24, 2026 1:49:05 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Sep 24, 2026 1:49:05 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Sep 24, 2026 1:49:06 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260924 13:49:07.259344 13911 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.13.149.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.13.149.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.13.149.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790257629041645-13911-0/krb5kdc/rangerkms_127.13.149.212@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.13.149.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790257629041645-13911-0/krb5kdc/rangerkms_127.13.149.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.13.149.212@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.13.149.212@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.13.149.212@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790257629041645-13911-0/krb5kdc/HTTP_127.13.149.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.13.149.212@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790257629041645-13911-0/krb5kdc/HTTP_127.13.149.212@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Sep 24 13:49:07 dist-test-slave-sjl2 krb5kdc[24759](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257747, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260924 13:49:07.344429 13911 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260924 13:49:07.353327 13911 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260924 13:49:08.280112 13911 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260924 13:49:08.280236 13911 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790257629041645-13911-0/minicluster-data/ranger-kms
I20260924 13:49:08.280267 13911 mini_ranger_kms.cc:227] Using postgres at 127.13.149.212:34565
2026-09-24 13:49:08,444 [I] DB FLAVOR :POSTGRES
2026-09-24 13:49:08,444 [I] --------- Verifying Ranger DB connection ---------
2026-09-24 13:49:08,445 [I] Checking connection
2026-09-24 13:49:08,445 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJN9WRF/build/release/bin/postgresql.jar:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790257629041645-13911-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.149.212:34565/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-09-24 13:49:08,693 [I] connection success
2026-09-24 13:49:08,693 [I] --------- Verifying Ranger DB tables ---------
2026-09-24 13:49:08,693 [I] Verifying table ranger_masterkey in database rangerkms
2026-09-24 13:49:08,693 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJN9WRF/build/release/bin/postgresql.jar:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790257629041645-13911-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.149.212:34565/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-09-24 13:49:08,935 [I] Table ranger_masterkey does not exist in database rangerkms
2026-09-24 13:49:08,936 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-24 13:49:08,936 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-09-24 13:49:08,936 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJN9WRF/build/release/bin/postgresql.jar:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790257629041645-13911-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.149.212:34565/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790257629041645-13911-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-09-24 13:49:09,191 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260924 13:49:09.654456 13911 mini_ranger_kms.cc:310] Created kms service
I20260924 13:49:09.861734 13911 mini_ranger_kms.cc:326] Created kudu user
I20260924 13:49:09.905112 13911 mini_ranger_kms.cc:343] Created rangerkms user
Sep 24 13:49:09 dist-test-slave-sjl2 krb5kdc[24759](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257747, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.13.149.212@KRBTEST.COM
I20260924 13:49:10.335608 13911 mini_ranger_kms.cc:384] Added ranger policy
I20260924 13:49:10.335732 13911 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790257629041645-13911-0/minicluster-data/ranger-kms:/tmp/dist-test-taskJN9WRF/build/release/bin/postgresql.jar:/tmp/dist-test-taskJN9WRF/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskJN9WRF/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260924 13:49:10.335754 13911 mini_ranger_kms.cc:240] Using host: 127.13.149.212
I20260924 13:49:10.339610 13911 mini_ranger_kms.cc:276] Ranger KMS PID: 25152
I20260924 13:49:10.339720 13911 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.13.149.212:56775
Sep 24, 2026 1:49:11 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 24, 2026 1:49:11 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 24, 2026 1:49:11 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790257629041645-13911-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Sep 24, 2026 1:49:11 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790257629041645-13911-0/minicluster-data/ranger-kms/ews/webapp] .....
Sep 24, 2026 1:49:11 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790257629041645-13911-0/minicluster-data/ranger-kms/ews/webapp].
Sep 24, 2026 1:49:11 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260924 13:49:12.412053 13911 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.068s user 0.008s sys 0.139s
I20260924 13:49:12.412155 13911 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260924 13:49:12.412245 13911 mini_ranger_kms.cc:401] 127.13.149.212:56775/kms/v1/keys
Sep 24 13:49:12 dist-test-slave-sjl2 krb5kdc[24759](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790257752, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.13.149.212@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Sep 24 13:49:13 dist-test-slave-sjl2 krb5kdc[24759](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257752, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.13.149.212@KRBTEST.COM for HTTP/127.13.149.212@KRBTEST.COM
Sep 24 13:49:13 dist-test-slave-sjl2 krb5kdc[24759](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257752, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.13.149.212@KRBTEST.COM for HTTP/127.13.149.212@KRBTEST.COM
Sep 24 13:49:15 dist-test-slave-sjl2 krb5kdc[24759](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257755, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.13.149.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.149.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.149.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.254.keytab.
Entry for principal kudu/127.13.149.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.254.keytab.
I20260924 13:49:15.872241 13911 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJN9WRF/build/release/bin/kudu
/tmp/dist-test-taskJN9WRF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790257629041645-13911-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790257629041645-13911-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790257629041645-13911-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790257629041645-13911-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.149.254:34083
--webserver_interface=127.13.149.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.254.keytab
--principal=kudu/127.13.149.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.149.212:42495
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.149.254:34083
--ranger_config_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790257629041645-13911-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.13.149.212:56775/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790257629041645-13911-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790257629041645-13911-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790257629041645-13911-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 13:49:15.975924 25203 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 13:49:15.976135 25203 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 13:49:15.976162 25203 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 13:49:15.977370 25203 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 13:49:15.977412 25203 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 13:49:15.977432 25203 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 13:49:15.977448 25203 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 13:49:15.977468 25203 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 13:49:15.979036 25203 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.149.212:42495
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790257629041645-13911-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790257629041645-13911-0/minicluster-data/master-0/wal
--ranger_kms_url=127.13.149.212:56775/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.13.149.254:34083
--ranger_config_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790257629041645-13911-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.254.keytab
--principal=kudu/127.13.149.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.149.254:34083
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790257629041645-13911-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.149.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25203
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790257629041645-13911-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 12:29:37 UTC on ccycloud.quasar-iqzbxn.root.comops.site
build id 4670106
I20260924 13:49:15.979274 25203 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 13:49:15.979478 25203 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 13:49:15.984160 25209 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 13:49:15.984337 25211 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 13:49:15.984210 25208 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 13:49:15.984637 25203 server_base.cc:1034] running on GCE node
I20260924 13:49:15.984855 25203 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 13:49:15.985141 25203 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 13:49:15.986303 25203 hybrid_clock.cc:648] HybridClock initialized: now 1790257755986291 us; error 27 us; skew 500 ppm
Sep 24 13:49:15 dist-test-slave-sjl2 krb5kdc[24759](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257755, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.149.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 13:49:15.988713 25203 init.cc:377] Logged in from keytab as kudu/127.13.149.254@KRBTEST.COM (short username kudu)
I20260924 13:49:15.989174 25203 webserver.cc:460] Webserver started at http://127.13.149.254:37671/ using document root <none> and password file <none>
I20260924 13:49:15.989382 25203 fs_manager.cc:359] Metadata directory not provided
I20260924 13:49:15.989420 25203 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 13:49:15.989543 25203 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 24 13:49:15 dist-test-slave-sjl2 krb5kdc[24759](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257755, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.149.254@KRBTEST.COM for HTTP/127.13.149.212@KRBTEST.COM
I20260924 13:49:16.053491 25203 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790257629041645-13911-0/minicluster-data/master-0/data/instance:
uuid: "5c231d9794ba4625856b262d5d874e35"
format_stamp: "Formatted at 2026-09-24 13:49:16 on dist-test-slave-sjl2"
server_key: "5dd71dea8912a2460e98d0529a38214f"
server_key_iv: "a728f0716ab8d5a8bdf29b38dcf26db8"
server_key_version: "kuduclusterkey@0"
I20260924 13:49:16.053920 25203 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790257629041645-13911-0/minicluster-data/master-0/wal/instance:
uuid: "5c231d9794ba4625856b262d5d874e35"
format_stamp: "Formatted at 2026-09-24 13:49:16 on dist-test-slave-sjl2"
server_key: "5dd71dea8912a2460e98d0529a38214f"
server_key_iv: "a728f0716ab8d5a8bdf29b38dcf26db8"
server_key_version: "kuduclusterkey@0"
I20260924 13:49:16.055388 25203 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260924 13:49:16.069886 25218 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 13:49:16.070148 25203 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260924 13:49:16.070271 25203 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790257629041645-13911-0/minicluster-data/master-0/data,/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790257629041645-13911-0/minicluster-data/master-0/wal
uuid: "5c231d9794ba4625856b262d5d874e35"
format_stamp: "Formatted at 2026-09-24 13:49:16 on dist-test-slave-sjl2"
server_key: "5dd71dea8912a2460e98d0529a38214f"
server_key_iv: "a728f0716ab8d5a8bdf29b38dcf26db8"
server_key_version: "kuduclusterkey@0"
I20260924 13:49:16.070381 25203 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790257629041645-13911-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790257629041645-13911-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790257629041645-13911-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 13:49:16.087296 25203 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 13:49:16.089265 25203 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 13:49:16.089401 25203 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 13:49:16.093781 25203 rpc_server.cc:307] RPC server started. Bound to: 127.13.149.254:34083
I20260924 13:49:16.093817 25270 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.149.254:34083 every 8 connection(s)
I20260924 13:49:16.094149 25203 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790257629041645-13911-0/minicluster-data/master-0/data/info.pb
I20260924 13:49:16.094918 25271 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 13:49:16.098039 25271 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5c231d9794ba4625856b262d5d874e35: Bootstrap starting.
I20260924 13:49:16.098796 25271 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5c231d9794ba4625856b262d5d874e35: Neither blocks nor log segments found. Creating new log.
I20260924 13:49:16.098870 13911 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJN9WRF/build/release/bin/kudu as pid 25203
I20260924 13:49:16.098973 13911 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790257629041645-13911-0/minicluster-data/master-0/wal/instance
I20260924 13:49:16.099109 25271 log.cc:824] T 00000000000000000000000000000000 P 5c231d9794ba4625856b262d5d874e35: Log is configured to *not* fsync() on all Append() calls
I20260924 13:49:16.099148 13911 external_mini_cluster.cc:1428] Setting key 77fd37c0a338886c24b2fa78b0120b65
I20260924 13:49:16.099920 25271 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5c231d9794ba4625856b262d5d874e35: No bootstrap required, opened a new log
I20260924 13:49:16.102283 25271 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5c231d9794ba4625856b262d5d874e35 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c231d9794ba4625856b262d5d874e35" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 34083 } }
I20260924 13:49:16.102470 25271 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5c231d9794ba4625856b262d5d874e35 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 13:49:16.102504 25271 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5c231d9794ba4625856b262d5d874e35 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c231d9794ba4625856b262d5d874e35, State: Initialized, Role: FOLLOWER
I20260924 13:49:16.102681 25271 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5c231d9794ba4625856b262d5d874e35 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c231d9794ba4625856b262d5d874e35" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 34083 } }
I20260924 13:49:16.102778 25271 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5c231d9794ba4625856b262d5d874e35 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 13:49:16.102837 25271 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5c231d9794ba4625856b262d5d874e35 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 13:49:16.102927 25271 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5c231d9794ba4625856b262d5d874e35 [term 0 FOLLOWER]: Advancing to term 1
I20260924 13:49:16.103658 25271 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5c231d9794ba4625856b262d5d874e35 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c231d9794ba4625856b262d5d874e35" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 34083 } }
I20260924 13:49:16.103777 25271 leader_election.cc:302] T 00000000000000000000000000000000 P 5c231d9794ba4625856b262d5d874e35 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5c231d9794ba4625856b262d5d874e35; no voters:
I20260924 13:49:16.103932 25271 leader_election.cc:288] T 00000000000000000000000000000000 P 5c231d9794ba4625856b262d5d874e35 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 13:49:16.103982 25276 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5c231d9794ba4625856b262d5d874e35 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 13:49:16.104271 25271 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5c231d9794ba4625856b262d5d874e35 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 13:49:16.104503 25276 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5c231d9794ba4625856b262d5d874e35 [term 1 LEADER]: Becoming Leader. State: Replica: 5c231d9794ba4625856b262d5d874e35, State: Running, Role: LEADER
Sep 24 13:49:16 dist-test-slave-sjl2 krb5kdc[24759](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257755, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.149.254@KRBTEST.COM
I20260924 13:49:16.106267 25276 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5c231d9794ba4625856b262d5d874e35 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c231d9794ba4625856b262d5d874e35" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 34083 } }
I20260924 13:49:16.106596 25276 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5c231d9794ba4625856b262d5d874e35 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5c231d9794ba4625856b262d5d874e35. Latest consensus state: current_term: 1 leader_uuid: "5c231d9794ba4625856b262d5d874e35" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c231d9794ba4625856b262d5d874e35" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 34083 } } }
I20260924 13:49:16.107079 25278 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5c231d9794ba4625856b262d5d874e35 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5c231d9794ba4625856b262d5d874e35" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c231d9794ba4625856b262d5d874e35" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 34083 } } }
I20260924 13:49:16.107342 25278 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5c231d9794ba4625856b262d5d874e35 [sys.catalog]: This master's current role is: LEADER
I20260924 13:49:16.107236 25271 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790257629041645-13911-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260924 13:49:16.107297 25276 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5c231d9794ba4625856b262d5d874e35 [sys.catalog]: This master's current role is: LEADER
I20260924 13:49:16.107615 25290 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 13:49:16.108141 25290 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 13:49:16.110183 25275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:49:16.099658 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52770 (local address 127.13.149.254:34083)
0924 13:49:16.099843 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0924 13:49:16.099853 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0924 13:49:16.099886 (+ 33us) server_negotiation.cc:407] Connection header received
0924 13:49:16.100447 (+ 561us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:49:16.100456 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:49:16.100521 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:49:16.100669 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:49:16.101469 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:49:16.102728 (+ 1259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:49:16.103528 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:49:16.104071 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:49:16.104168 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:49:16.106720 (+ 2552us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:49:16.106724 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:49:16.106725 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 13:49:16.106736 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:49:16.108568 (+ 1832us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:49:16.108995 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:49:16.108998 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:49:16.108999 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:49:16.109096 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:49:16.109304 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:49:16.109305 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:49:16.109305 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:49:16.109597 (+ 292us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:49:16.109740 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0924 13:49:16.109877 (+ 137us) server_negotiation.cc:299] Negotiation successful
0924 13:49:16.109992 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":141,"thread_start_us":78,"threads_started":1}
I20260924 13:49:16.111171 25290 catalog_manager.cc:1347] Generated new cluster ID: b303d45f148c4c2abac23d5d1eccbbac
I20260924 13:49:16.111230 25290 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 13:49:16.124500 25290 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 13:49:16.125358 25290 catalog_manager.cc:1504] Loading token signing keys...
I20260924 13:49:16.131970 25290 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5c231d9794ba4625856b262d5d874e35: Generated new TSK 0
I20260924 13:49:16.132263 25290 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.13.149.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.149.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.149.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.193.keytab.
Entry for principal kudu/127.13.149.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.193.keytab.
I20260924 13:49:16.870149 13911 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJN9WRF/build/release/bin/kudu
/tmp/dist-test-taskJN9WRF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790257629041645-13911-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790257629041645-13911-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790257629041645-13911-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790257629041645-13911-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.13.149.212:56775/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.149.193:0
--local_ip_for_outbound_sockets=127.13.149.193
--webserver_interface=127.13.149.193
--webserver_port=0
--tserver_master_addrs=127.13.149.254:34083
--keytab_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.193.keytab
--principal=kudu/127.13.149.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.149.212:42495
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790257629041645-13911-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790257629041645-13911-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790257629041645-13911-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 13:49:16.979360 25316 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 13:49:16.979562 25316 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 13:49:16.979596 25316 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 13:49:16.980764 25316 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 13:49:16.980804 25316 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 13:49:16.980839 25316 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.149.193
I20260924 13:49:16.982431 25316 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.149.212:42495
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790257629041645-13911-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790257629041645-13911-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.13.149.212:56775/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.193.keytab
--principal=kudu/127.13.149.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.149.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790257629041645-13911-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.149.193
--webserver_port=0
--tserver_master_addrs=127.13.149.254:34083
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25316
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.149.193
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790257629041645-13911-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 12:29:37 UTC on ccycloud.quasar-iqzbxn.root.comops.site
build id 4670106
I20260924 13:49:16.982663 25316 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 13:49:16.982995 25316 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 13:49:16.989154 25321 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 13:49:16.989270 25322 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 13:49:16.989307 25316 server_base.cc:1034] running on GCE node
W20260924 13:49:16.989142 25324 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 13:49:16.989598 25316 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 13:49:16.989843 25316 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 13:49:16.990981 25316 hybrid_clock.cc:648] HybridClock initialized: now 1790257756990968 us; error 31 us; skew 500 ppm
Sep 24 13:49:16 dist-test-slave-sjl2 krb5kdc[24759](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257756, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.149.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 13:49:16.993428 25316 init.cc:377] Logged in from keytab as kudu/127.13.149.193@KRBTEST.COM (short username kudu)
I20260924 13:49:16.993866 25316 webserver.cc:460] Webserver started at http://127.13.149.193:36619/ using document root <none> and password file <none>
I20260924 13:49:16.994081 25316 fs_manager.cc:359] Metadata directory not provided
I20260924 13:49:16.994127 25316 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 13:49:16.994257 25316 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 24 13:49:16 dist-test-slave-sjl2 krb5kdc[24759](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257756, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.149.193@KRBTEST.COM for HTTP/127.13.149.212@KRBTEST.COM
I20260924 13:49:17.013885 25316 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790257629041645-13911-0/minicluster-data/ts-0/data/instance:
uuid: "c735360ad0164088af9122a62b4a7b1f"
format_stamp: "Formatted at 2026-09-24 13:49:17 on dist-test-slave-sjl2"
server_key: "688d2a9b0d8c20195cdc92bc67685e86"
server_key_iv: "7b2356241c8f28c87b1cec9eb40c7d62"
server_key_version: "kuduclusterkey@0"
I20260924 13:49:17.014357 25316 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790257629041645-13911-0/minicluster-data/ts-0/wal/instance:
uuid: "c735360ad0164088af9122a62b4a7b1f"
format_stamp: "Formatted at 2026-09-24 13:49:17 on dist-test-slave-sjl2"
server_key: "688d2a9b0d8c20195cdc92bc67685e86"
server_key_iv: "7b2356241c8f28c87b1cec9eb40c7d62"
server_key_version: "kuduclusterkey@0"
I20260924 13:49:17.016115 25316 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20260924 13:49:17.037217 25333 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 13:49:17.037458 25316 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260924 13:49:17.037539 25316 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790257629041645-13911-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790257629041645-13911-0/minicluster-data/ts-0/wal
uuid: "c735360ad0164088af9122a62b4a7b1f"
format_stamp: "Formatted at 2026-09-24 13:49:17 on dist-test-slave-sjl2"
server_key: "688d2a9b0d8c20195cdc92bc67685e86"
server_key_iv: "7b2356241c8f28c87b1cec9eb40c7d62"
server_key_version: "kuduclusterkey@0"
I20260924 13:49:17.037627 25316 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790257629041645-13911-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790257629041645-13911-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790257629041645-13911-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 13:49:17.056802 25316 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 13:49:17.059135 25316 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 13:49:17.059341 25316 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 13:49:17.059600 25316 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 13:49:17.060004 25316 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 13:49:17.060086 25316 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 13:49:17.060178 25316 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 13:49:17.060246 25316 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 13:49:17.066720 25316 rpc_server.cc:307] RPC server started. Bound to: 127.13.149.193:45131
I20260924 13:49:17.066869 25446 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.149.193:45131 every 8 connection(s)
I20260924 13:49:17.067191 25316 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790257629041645-13911-0/minicluster-data/ts-0/data/info.pb
I20260924 13:49:17.076840 13911 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJN9WRF/build/release/bin/kudu as pid 25316
I20260924 13:49:17.076977 13911 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790257629041645-13911-0/minicluster-data/ts-0/wal/instance
I20260924 13:49:17.077268 13911 external_mini_cluster.cc:1428] Setting key 42a700b127a60a3376f6b8964d4274ac
Sep 24 13:49:17 dist-test-slave-sjl2 krb5kdc[24759](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257756, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.149.193@KRBTEST.COM for kudu/127.13.149.254@KRBTEST.COM
I20260924 13:49:17.082146 25450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:49:17.067873 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:34083 (local address 127.13.149.193:43173)
0924 13:49:17.073213 (+ 5340us) negotiation.cc:107] Waiting for socket to connect
0924 13:49:17.073253 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0924 13:49:17.073809 (+ 556us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:49:17.074058 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:49:17.074063 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:49:17.074302 (+ 239us) client_negotiation.cc:189] Negotiated authn=SASL
0924 13:49:17.074499 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:49:17.074504 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:49:17.075242 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:49:17.075243 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:49:17.076070 (+ 827us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:49:17.076073 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:49:17.076421 (+ 348us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:49:17.076423 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:49:17.076538 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:49:17.077112 (+ 574us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 13:49:17.077122 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0924 13:49:17.078649 (+ 1527us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 13:49:17.080528 (+ 1879us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 13:49:17.080534 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 13:49:17.080536 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 13:49:17.080824 (+ 288us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 13:49:17.081073 (+ 249us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 13:49:17.081074 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 13:49:17.081075 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 13:49:17.081197 (+ 122us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 13:49:17.081465 (+ 268us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 13:49:17.081470 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 13:49:17.081622 (+ 152us) client_negotiation.cc:769] Sending connection context
0924 13:49:17.081752 (+ 130us) client_negotiation.cc:240] Negotiation successful
0924 13:49:17.081911 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5301,"thread_start_us":78,"threads_started":1}
I20260924 13:49:17.082602 25449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:49:17.067897 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.149.193:43173 (local address 127.13.149.254:34083)
0924 13:49:17.073146 (+ 5249us) server_negotiation.cc:206] Beginning negotiation
0924 13:49:17.073148 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:49:17.073348 (+ 200us) server_negotiation.cc:407] Connection header received
0924 13:49:17.073903 (+ 555us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:49:17.073905 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:49:17.073919 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:49:17.073986 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:49:17.074637 (+ 651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:49:17.075152 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:49:17.076145 (+ 993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:49:17.076352 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:49:17.076406 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:49:17.078758 (+ 2352us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:49:17.078759 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:49:17.078760 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 13:49:17.078767 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:49:17.080438 (+ 1671us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:49:17.080905 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:49:17.080907 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:49:17.080907 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:49:17.081007 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:49:17.081261 (+ 254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:49:17.081262 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:49:17.081262 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:49:17.081387 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:49:17.081441 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0924 13:49:17.082414 (+ 973us) server_negotiation.cc:299] Negotiation successful
0924 13:49:17.082498 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5213,"thread_start_us":64,"threads_started":1}
I20260924 13:49:17.082957 25447 heartbeater.cc:347] Connected to a master server at 127.13.149.254:34083
I20260924 13:49:17.083043 25447 heartbeater.cc:464] Registering TS with master...
I20260924 13:49:17.083220 25447 heartbeater.cc:511] Master 127.13.149.254:34083 requested a full tablet report, sending...
I20260924 13:49:17.083755 25235 ts_manager.cc:194] Registered new tserver with Master: c735360ad0164088af9122a62b4a7b1f (127.13.149.193:45131)
I20260924 13:49:17.084600 25235 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.149.193@KRBTEST.COM'} at 127.13.149.193:43173
WARNING: no policy specified for kudu/127.13.149.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.149.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.149.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.194.keytab.
Entry for principal kudu/127.13.149.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.194.keytab.
I20260924 13:49:17.144555 13911 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJN9WRF/build/release/bin/kudu
/tmp/dist-test-taskJN9WRF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790257629041645-13911-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790257629041645-13911-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790257629041645-13911-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790257629041645-13911-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.13.149.212:56775/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.149.194:0
--local_ip_for_outbound_sockets=127.13.149.194
--webserver_interface=127.13.149.194
--webserver_port=0
--tserver_master_addrs=127.13.149.254:34083
--keytab_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.194.keytab
--principal=kudu/127.13.149.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.149.212:42495
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790257629041645-13911-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790257629041645-13911-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790257629041645-13911-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 24 13:49:17 dist-test-slave-sjl2 krb5kdc[24759](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790257757, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.149.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260924 13:49:17.275189 25455 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 13:49:17.275538 25455 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 13:49:17.275619 25455 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 13:49:17.277367 25455 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 13:49:17.277464 25455 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 13:49:17.277539 25455 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.149.194
I20260924 13:49:17.279616 25455 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.149.212:42495
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790257629041645-13911-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790257629041645-13911-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.13.149.212:56775/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.194.keytab
--principal=kudu/127.13.149.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.149.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790257629041645-13911-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.149.194
--webserver_port=0
--tserver_master_addrs=127.13.149.254:34083
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25455
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.149.194
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790257629041645-13911-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 12:29:37 UTC on ccycloud.quasar-iqzbxn.root.comops.site
build id 4670106
I20260924 13:49:17.280107 25455 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 13:49:17.280404 25455 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 13:49:17.286499 25461 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 13:49:17.287060 25460 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 13:49:17.287355 25463 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 13:49:17.287549 25455 server_base.cc:1034] running on GCE node
I20260924 13:49:17.287752 25455 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 13:49:17.287986 25455 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 13:49:17.289173 25455 hybrid_clock.cc:648] HybridClock initialized: now 1790257757289150 us; error 38 us; skew 500 ppm
Sep 24 13:49:17 dist-test-slave-sjl2 krb5kdc[24759](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257757, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.149.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 13:49:17.292188 25455 init.cc:377] Logged in from keytab as kudu/127.13.149.194@KRBTEST.COM (short username kudu)
I20260924 13:49:17.292685 25455 webserver.cc:460] Webserver started at http://127.13.149.194:45069/ using document root <none> and password file <none>
I20260924 13:49:17.292961 25455 fs_manager.cc:359] Metadata directory not provided
I20260924 13:49:17.293011 25455 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 13:49:17.293124 25455 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 24 13:49:17 dist-test-slave-sjl2 krb5kdc[24759](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257757, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.149.194@KRBTEST.COM for HTTP/127.13.149.212@KRBTEST.COM
I20260924 13:49:17.306603 25455 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790257629041645-13911-0/minicluster-data/ts-1/data/instance:
uuid: "8a8f26947ec54029bca8c8567da11cf3"
format_stamp: "Formatted at 2026-09-24 13:49:17 on dist-test-slave-sjl2"
server_key: "2ab39d38fa2760efdd4e597891023d22"
server_key_iv: "fea56bfdd71c1889b0a888953b8be20e"
server_key_version: "kuduclusterkey@0"
I20260924 13:49:17.307149 25455 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790257629041645-13911-0/minicluster-data/ts-1/wal/instance:
uuid: "8a8f26947ec54029bca8c8567da11cf3"
format_stamp: "Formatted at 2026-09-24 13:49:17 on dist-test-slave-sjl2"
server_key: "2ab39d38fa2760efdd4e597891023d22"
server_key_iv: "fea56bfdd71c1889b0a888953b8be20e"
server_key_version: "kuduclusterkey@0"
I20260924 13:49:17.313383 25455 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.001s sys 0.000s
I20260924 13:49:17.328065 25471 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 13:49:17.328341 25455 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260924 13:49:17.328439 25455 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790257629041645-13911-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790257629041645-13911-0/minicluster-data/ts-1/wal
uuid: "8a8f26947ec54029bca8c8567da11cf3"
format_stamp: "Formatted at 2026-09-24 13:49:17 on dist-test-slave-sjl2"
server_key: "2ab39d38fa2760efdd4e597891023d22"
server_key_iv: "fea56bfdd71c1889b0a888953b8be20e"
server_key_version: "kuduclusterkey@0"
I20260924 13:49:17.328559 25455 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790257629041645-13911-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790257629041645-13911-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790257629041645-13911-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 13:49:17.350289 25455 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 13:49:17.352540 25455 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 13:49:17.352743 25455 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 13:49:17.353021 25455 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 13:49:17.353397 25455 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 13:49:17.353477 25455 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 13:49:17.353542 25455 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 13:49:17.353595 25455 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 13:49:17.359485 25455 rpc_server.cc:307] RPC server started. Bound to: 127.13.149.194:38515
I20260924 13:49:17.360040 25455 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790257629041645-13911-0/minicluster-data/ts-1/data/info.pb
Sep 24 13:49:17 dist-test-slave-sjl2 krb5kdc[24759](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257757, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.149.194@KRBTEST.COM for kudu/127.13.149.254@KRBTEST.COM
I20260924 13:49:17.370476 13911 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJN9WRF/build/release/bin/kudu as pid 25455
I20260924 13:49:17.370580 13911 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790257629041645-13911-0/minicluster-data/ts-1/wal/instance
I20260924 13:49:17.370786 13911 external_mini_cluster.cc:1428] Setting key 0099b712d00d4ac5f7647352bb281708
I20260924 13:49:17.362694 25584 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.149.194:38515 every 8 connection(s)
I20260924 13:49:17.376636 25449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:49:17.363633 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.149.194:52081 (local address 127.13.149.254:34083)
0924 13:49:17.363714 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0924 13:49:17.363736 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0924 13:49:17.363993 (+ 257us) server_negotiation.cc:407] Connection header received
0924 13:49:17.364684 (+ 691us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:49:17.364685 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:49:17.364700 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:49:17.364741 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:49:17.365566 (+ 825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:49:17.366321 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:49:17.367784 (+ 1463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:49:17.368103 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:49:17.368148 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:49:17.371440 (+ 3292us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:49:17.371441 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:49:17.371442 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 13:49:17.371450 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:49:17.374799 (+ 3349us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:49:17.375478 (+ 679us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:49:17.375479 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:49:17.375480 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:49:17.375544 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:49:17.375859 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:49:17.375861 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:49:17.375861 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:49:17.376009 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:49:17.376053 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0924 13:49:17.376439 (+ 386us) server_negotiation.cc:299] Negotiation successful
0924 13:49:17.376535 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260924 13:49:17.377467 25587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:49:17.363337 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:34083 (local address 127.13.149.194:52081)
0924 13:49:17.363861 (+ 524us) negotiation.cc:107] Waiting for socket to connect
0924 13:49:17.363907 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0924 13:49:17.364560 (+ 653us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:49:17.364945 (+ 385us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:49:17.364950 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:49:17.365189 (+ 239us) client_negotiation.cc:189] Negotiated authn=SASL
0924 13:49:17.365461 (+ 272us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:49:17.365467 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:49:17.366439 (+ 972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:49:17.366440 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:49:17.367674 (+ 1234us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:49:17.367677 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:49:17.368236 (+ 559us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:49:17.368237 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:49:17.368378 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:49:17.368978 (+ 600us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 13:49:17.368988 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0924 13:49:17.371335 (+ 2347us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 13:49:17.375050 (+ 3715us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 13:49:17.375056 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 13:49:17.375059 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0924 13:49:17.375382 (+ 323us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 13:49:17.375619 (+ 237us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 13:49:17.375619 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 13:49:17.375620 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 13:49:17.375776 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 13:49:17.376099 (+ 323us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 13:49:17.376105 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 13:49:17.376277 (+ 172us) client_negotiation.cc:769] Sending connection context
0924 13:49:17.377027 (+ 750us) client_negotiation.cc:240] Negotiation successful
0924 13:49:17.377227 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":482,"thread_start_us":86,"threads_started":1}
I20260924 13:49:17.378131 25585 heartbeater.cc:347] Connected to a master server at 127.13.149.254:34083
I20260924 13:49:17.378229 25585 heartbeater.cc:464] Registering TS with master...
I20260924 13:49:17.378427 25585 heartbeater.cc:511] Master 127.13.149.254:34083 requested a full tablet report, sending...
I20260924 13:49:17.378883 25235 ts_manager.cc:194] Registered new tserver with Master: 8a8f26947ec54029bca8c8567da11cf3 (127.13.149.194:38515)
I20260924 13:49:17.379367 25235 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.149.194@KRBTEST.COM'} at 127.13.149.194:52081
WARNING: no policy specified for kudu/127.13.149.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.149.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.149.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.195.keytab.
Entry for principal kudu/127.13.149.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.195.keytab.
I20260924 13:49:17.408383 13911 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJN9WRF/build/release/bin/kudu
/tmp/dist-test-taskJN9WRF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790257629041645-13911-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790257629041645-13911-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790257629041645-13911-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790257629041645-13911-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.13.149.212:56775/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.149.195:0
--local_ip_for_outbound_sockets=127.13.149.195
--webserver_interface=127.13.149.195
--webserver_port=0
--tserver_master_addrs=127.13.149.254:34083
--keytab_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.195.keytab
--principal=kudu/127.13.149.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.149.212:42495
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790257629041645-13911-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790257629041645-13911-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790257629041645-13911-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 13:49:17.528137 25590 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 13:49:17.528470 25590 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 13:49:17.528548 25590 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 13:49:17.530205 25590 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 13:49:17.530300 25590 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 13:49:17.530376 25590 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.149.195
I20260924 13:49:17.532362 25590 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.149.212:42495
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790257629041645-13911-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790257629041645-13911-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.13.149.212:56775/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.195.keytab
--principal=kudu/127.13.149.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.149.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790257629041645-13911-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.149.195
--webserver_port=0
--tserver_master_addrs=127.13.149.254:34083
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25590
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.149.195
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790257629041645-13911-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 12:29:37 UTC on ccycloud.quasar-iqzbxn.root.comops.site
build id 4670106
I20260924 13:49:17.532982 25590 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 13:49:17.533281 25590 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 13:49:17.539651 25595 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 13:49:17.539619 25596 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 13:49:17.540105 25590 server_base.cc:1034] running on GCE node
W20260924 13:49:17.541065 25598 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 13:49:17.541304 25590 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 13:49:17.541572 25590 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 13:49:17.542734 25590 hybrid_clock.cc:648] HybridClock initialized: now 1790257757542704 us; error 38 us; skew 500 ppm
Sep 24 13:49:17 dist-test-slave-sjl2 krb5kdc[24759](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257757, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.149.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 13:49:17.545938 25590 init.cc:377] Logged in from keytab as kudu/127.13.149.195@KRBTEST.COM (short username kudu)
I20260924 13:49:17.546478 25590 webserver.cc:460] Webserver started at http://127.13.149.195:36175/ using document root <none> and password file <none>
I20260924 13:49:17.546736 25590 fs_manager.cc:359] Metadata directory not provided
I20260924 13:49:17.546792 25590 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 13:49:17.546958 25590 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 24 13:49:17 dist-test-slave-sjl2 krb5kdc[24759](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257757, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.149.195@KRBTEST.COM for HTTP/127.13.149.212@KRBTEST.COM
I20260924 13:49:17.561120 25590 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790257629041645-13911-0/minicluster-data/ts-2/data/instance:
uuid: "14d6435e29a641c784cf961d09c28019"
format_stamp: "Formatted at 2026-09-24 13:49:17 on dist-test-slave-sjl2"
server_key: "afee0eae86f7c4477591a339dfe29494"
server_key_iv: "b78e27fe67c770fc2a1b4965b7dd71b0"
server_key_version: "kuduclusterkey@0"
I20260924 13:49:17.561547 25590 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790257629041645-13911-0/minicluster-data/ts-2/wal/instance:
uuid: "14d6435e29a641c784cf961d09c28019"
format_stamp: "Formatted at 2026-09-24 13:49:17 on dist-test-slave-sjl2"
server_key: "afee0eae86f7c4477591a339dfe29494"
server_key_iv: "b78e27fe67c770fc2a1b4965b7dd71b0"
server_key_version: "kuduclusterkey@0"
I20260924 13:49:17.580303 25590 fs_manager.cc:691] Time spent creating directory manager: real 0.019s user 0.002s sys 0.000s
I20260924 13:49:17.602111 25605 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 13:49:17.602411 25590 fs_manager.cc:725] Time spent opening block manager: real 0.010s user 0.000s sys 0.001s
I20260924 13:49:17.602506 25590 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790257629041645-13911-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790257629041645-13911-0/minicluster-data/ts-2/wal
uuid: "14d6435e29a641c784cf961d09c28019"
format_stamp: "Formatted at 2026-09-24 13:49:17 on dist-test-slave-sjl2"
server_key: "afee0eae86f7c4477591a339dfe29494"
server_key_iv: "b78e27fe67c770fc2a1b4965b7dd71b0"
server_key_version: "kuduclusterkey@0"
I20260924 13:49:17.602612 25590 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790257629041645-13911-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790257629041645-13911-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790257629041645-13911-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 13:49:17.625689 25590 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 13:49:17.628039 25590 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 13:49:17.628242 25590 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 13:49:17.628557 25590 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 13:49:17.628986 25590 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 13:49:17.629071 25590 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 13:49:17.629140 25590 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 13:49:17.629210 25590 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 13:49:17.635385 25590 rpc_server.cc:307] RPC server started. Bound to: 127.13.149.195:44457
I20260924 13:49:17.635927 25590 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790257629041645-13911-0/minicluster-data/ts-2/data/info.pb
I20260924 13:49:17.638710 25718 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.149.195:44457 every 8 connection(s)
I20260924 13:49:17.640517 13911 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJN9WRF/build/release/bin/kudu as pid 25590
I20260924 13:49:17.640619 13911 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790257629041645-13911-0/minicluster-data/ts-2/wal/instance
I20260924 13:49:17.640811 13911 external_mini_cluster.cc:1428] Setting key 85c42484acddee6d5fbb8913f5c8bebe
Sep 24 13:49:17 dist-test-slave-sjl2 krb5kdc[24759](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257757, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.149.195@KRBTEST.COM for kudu/127.13.149.254@KRBTEST.COM
I20260924 13:49:17.651504 25449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:49:17.639637 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.149.195:45705 (local address 127.13.149.254:34083)
0924 13:49:17.639683 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0924 13:49:17.639685 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:49:17.640117 (+ 432us) server_negotiation.cc:407] Connection header received
0924 13:49:17.641915 (+ 1798us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:49:17.641916 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:49:17.641928 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:49:17.641958 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:49:17.642786 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:49:17.643770 (+ 984us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:49:17.645280 (+ 1510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:49:17.645582 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:49:17.645623 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:49:17.648234 (+ 2611us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:49:17.648235 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:49:17.648236 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 13:49:17.648244 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:49:17.649925 (+ 1681us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:49:17.650418 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:49:17.650419 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:49:17.650420 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:49:17.650477 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:49:17.650756 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:49:17.650757 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:49:17.650757 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:49:17.650896 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:49:17.650938 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0924 13:49:17.651319 (+ 381us) server_negotiation.cc:299] Negotiation successful
0924 13:49:17.651403 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260924 13:49:17.652485 25721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:49:17.639343 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:34083 (local address 127.13.149.195:45705)
0924 13:49:17.639986 (+ 643us) negotiation.cc:107] Waiting for socket to connect
0924 13:49:17.640031 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0924 13:49:17.641797 (+ 1766us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:49:17.642149 (+ 352us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:49:17.642154 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:49:17.642400 (+ 246us) client_negotiation.cc:189] Negotiated authn=SASL
0924 13:49:17.642682 (+ 282us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:49:17.642688 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:49:17.643878 (+ 1190us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:49:17.643879 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:49:17.645183 (+ 1304us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:49:17.645186 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:49:17.645717 (+ 531us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:49:17.645718 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:49:17.645861 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:49:17.646455 (+ 594us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 13:49:17.646465 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0924 13:49:17.648105 (+ 1640us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 13:49:17.650019 (+ 1914us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 13:49:17.650025 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 13:49:17.650028 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0924 13:49:17.650331 (+ 303us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 13:49:17.650544 (+ 213us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 13:49:17.650545 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 13:49:17.650545 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 13:49:17.650676 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 13:49:17.650983 (+ 307us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 13:49:17.650989 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 13:49:17.651152 (+ 163us) client_negotiation.cc:769] Sending connection context
0924 13:49:17.651978 (+ 826us) client_negotiation.cc:240] Negotiation successful
0924 13:49:17.652144 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":599,"thread_start_us":91,"threads_started":1}
I20260924 13:49:17.653131 25719 heartbeater.cc:347] Connected to a master server at 127.13.149.254:34083
I20260924 13:49:17.653237 25719 heartbeater.cc:464] Registering TS with master...
I20260924 13:49:17.653419 25719 heartbeater.cc:511] Master 127.13.149.254:34083 requested a full tablet report, sending...
I20260924 13:49:17.653807 25235 ts_manager.cc:194] Registered new tserver with Master: 14d6435e29a641c784cf961d09c28019 (127.13.149.195:44457)
I20260924 13:49:17.654285 25235 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.149.195@KRBTEST.COM'} at 127.13.149.195:45705
I20260924 13:49:17.663275 13911 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260924 13:49:17.681563 25449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:49:17.666453 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52786 (local address 127.13.149.254:34083)
0924 13:49:17.666499 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0924 13:49:17.666501 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:49:17.673108 (+ 6607us) server_negotiation.cc:407] Connection header received
0924 13:49:17.673204 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:49:17.673205 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:49:17.673217 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:49:17.673277 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:49:17.674115 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:49:17.674824 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:49:17.676089 (+ 1265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:49:17.676348 (+ 259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:49:17.676389 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:49:17.678792 (+ 2403us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:49:17.678794 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:49:17.678794 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 13:49:17.678803 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:49:17.680345 (+ 1542us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:49:17.680715 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:49:17.680716 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:49:17.680716 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:49:17.680783 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:49:17.680991 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:49:17.680992 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:49:17.680992 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:49:17.681120 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:49:17.681192 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0924 13:49:17.681385 (+ 193us) server_negotiation.cc:299] Negotiation successful
0924 13:49:17.681467 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260924 13:49:17.683765 25235 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52786:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 13:49:17.686003 25235 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 13:49:17.702292 25736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:49:17.695885 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.193:45131 (local address 127.0.0.1:60446)
0924 13:49:17.697501 (+ 1616us) negotiation.cc:107] Waiting for socket to connect
0924 13:49:17.697545 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0924 13:49:17.697605 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:49:17.697963 (+ 358us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:49:17.697966 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:49:17.697968 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 13:49:17.698035 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:49:17.698037 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:49:17.699476 (+ 1439us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:49:17.699477 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:49:17.700696 (+ 1219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:49:17.700699 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:49:17.702005 (+ 1306us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:49:17.702006 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:49:17.702084 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:49:17.702097 (+ 13us) client_negotiation.cc:769] Sending connection context
0924 13:49:17.702154 (+ 57us) client_negotiation.cc:240] Negotiation successful
0924 13:49:17.702202 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1588,"thread_start_us":71,"threads_started":1}
I20260924 13:49:17.703080 25734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:49:17.696343 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60446 (local address 127.13.149.193:45131)
0924 13:49:17.697764 (+ 1421us) server_negotiation.cc:206] Beginning negotiation
0924 13:49:17.697768 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 13:49:17.697777 (+ 9us) server_negotiation.cc:407] Connection header received
0924 13:49:17.697809 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:49:17.697811 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:49:17.697827 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:49:17.697887 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 13:49:17.698101 (+ 214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:49:17.699401 (+ 1300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:49:17.700800 (+ 1399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:49:17.701854 (+ 1054us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:49:17.701919 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:49:17.701973 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0924 13:49:17.702962 (+ 989us) server_negotiation.cc:299] Negotiation successful
0924 13:49:17.703004 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1383,"thread_start_us":48,"threads_started":1}
I20260924 13:49:17.703863 25381 tablet_service.cc:1467] Processing CreateTablet for tablet f89b886e7d1c491f80c5d138382d726c (DEFAULT_TABLE table=test-table [id=9557c0b6bbfa4e5fac47b9ebff54660d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 13:49:17.704226 25381 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f89b886e7d1c491f80c5d138382d726c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 13:49:17.704952 25732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:49:17.696098 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.194:38515 (local address 127.0.0.1:53028)
0924 13:49:17.700980 (+ 4882us) negotiation.cc:107] Waiting for socket to connect
0924 13:49:17.700992 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0924 13:49:17.701049 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:49:17.701333 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:49:17.701334 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:49:17.701336 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 13:49:17.701376 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:49:17.701378 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:49:17.702618 (+ 1240us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:49:17.702619 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:49:17.703647 (+ 1028us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:49:17.703649 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:49:17.704731 (+ 1082us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:49:17.704732 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:49:17.704787 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:49:17.704790 (+ 3us) client_negotiation.cc:769] Sending connection context
0924 13:49:17.704830 (+ 40us) client_negotiation.cc:240] Negotiation successful
0924 13:49:17.704864 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4864,"thread_start_us":53,"threads_started":1}
I20260924 13:49:17.705164 25735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:49:17.696480 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53028 (local address 127.13.149.194:38515)
0924 13:49:17.697443 (+ 963us) server_negotiation.cc:206] Beginning negotiation
0924 13:49:17.697447 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 13:49:17.701106 (+ 3659us) server_negotiation.cc:407] Connection header received
0924 13:49:17.701152 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:49:17.701154 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:49:17.701171 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:49:17.701251 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 13:49:17.701446 (+ 195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:49:17.702421 (+ 975us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:49:17.703752 (+ 1331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:49:17.704433 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:49:17.704514 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:49:17.704562 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0924 13:49:17.705044 (+ 482us) server_negotiation.cc:299] Negotiation successful
0924 13:49:17.705088 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":933,"thread_start_us":63,"threads_started":1}
I20260924 13:49:17.705945 25519 tablet_service.cc:1467] Processing CreateTablet for tablet f89b886e7d1c491f80c5d138382d726c (DEFAULT_TABLE table=test-table [id=9557c0b6bbfa4e5fac47b9ebff54660d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 13:49:17.706293 25519 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f89b886e7d1c491f80c5d138382d726c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 13:49:17.714385 25739 tablet_bootstrap.cc:492] T f89b886e7d1c491f80c5d138382d726c P 8a8f26947ec54029bca8c8567da11cf3: Bootstrap starting.
I20260924 13:49:17.716490 25733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:49:17.696752 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.195:44457 (local address 127.0.0.1:43596)
0924 13:49:17.702519 (+ 5767us) negotiation.cc:107] Waiting for socket to connect
0924 13:49:17.702529 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 13:49:17.702568 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:49:17.704594 (+ 2026us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:49:17.704595 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:49:17.704597 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 13:49:17.704632 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:49:17.704634 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:49:17.714243 (+ 9609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:49:17.714245 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:49:17.715322 (+ 1077us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:49:17.715326 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:49:17.716240 (+ 914us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:49:17.716241 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:49:17.716305 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:49:17.716310 (+ 5us) client_negotiation.cc:769] Sending connection context
0924 13:49:17.716365 (+ 55us) client_negotiation.cc:240] Negotiation successful
0924 13:49:17.716411 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5752,"thread_start_us":56,"threads_started":1}
I20260924 13:49:17.716498 25737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:49:17.696976 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43596 (local address 127.13.149.195:44457)
0924 13:49:17.697365 (+ 389us) server_negotiation.cc:206] Beginning negotiation
0924 13:49:17.697370 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 13:49:17.702600 (+ 5230us) server_negotiation.cc:407] Connection header received
0924 13:49:17.702644 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:49:17.702646 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:49:17.702662 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:49:17.702725 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 13:49:17.712652 (+ 9927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:49:17.714142 (+ 1490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:49:17.715424 (+ 1282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:49:17.716146 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:49:17.716237 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:49:17.716298 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0924 13:49:17.716380 (+ 82us) server_negotiation.cc:299] Negotiation successful
0924 13:49:17.716416 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":349,"thread_start_us":75,"threads_started":1}
I20260924 13:49:17.717417 25653 tablet_service.cc:1467] Processing CreateTablet for tablet f89b886e7d1c491f80c5d138382d726c (DEFAULT_TABLE table=test-table [id=9557c0b6bbfa4e5fac47b9ebff54660d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 13:49:17.717749 25653 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f89b886e7d1c491f80c5d138382d726c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 13:49:17.721588 25738 tablet_bootstrap.cc:492] T f89b886e7d1c491f80c5d138382d726c P c735360ad0164088af9122a62b4a7b1f: Bootstrap starting.
I20260924 13:49:17.722405 25738 tablet_bootstrap.cc:654] T f89b886e7d1c491f80c5d138382d726c P c735360ad0164088af9122a62b4a7b1f: Neither blocks nor log segments found. Creating new log.
I20260924 13:49:17.722558 25739 tablet_bootstrap.cc:654] T f89b886e7d1c491f80c5d138382d726c P 8a8f26947ec54029bca8c8567da11cf3: Neither blocks nor log segments found. Creating new log.
I20260924 13:49:17.722941 25739 log.cc:824] T f89b886e7d1c491f80c5d138382d726c P 8a8f26947ec54029bca8c8567da11cf3: Log is configured to *not* fsync() on all Append() calls
I20260924 13:49:17.723505 25738 log.cc:824] T f89b886e7d1c491f80c5d138382d726c P c735360ad0164088af9122a62b4a7b1f: Log is configured to *not* fsync() on all Append() calls
I20260924 13:49:17.723889 25740 tablet_bootstrap.cc:492] T f89b886e7d1c491f80c5d138382d726c P 14d6435e29a641c784cf961d09c28019: Bootstrap starting.
I20260924 13:49:17.724880 25740 tablet_bootstrap.cc:654] T f89b886e7d1c491f80c5d138382d726c P 14d6435e29a641c784cf961d09c28019: Neither blocks nor log segments found. Creating new log.
I20260924 13:49:17.725231 25740 log.cc:824] T f89b886e7d1c491f80c5d138382d726c P 14d6435e29a641c784cf961d09c28019: Log is configured to *not* fsync() on all Append() calls
I20260924 13:49:17.726037 25739 tablet_bootstrap.cc:492] T f89b886e7d1c491f80c5d138382d726c P 8a8f26947ec54029bca8c8567da11cf3: No bootstrap required, opened a new log
I20260924 13:49:17.726152 25739 ts_tablet_manager.cc:1397] T f89b886e7d1c491f80c5d138382d726c P 8a8f26947ec54029bca8c8567da11cf3: Time spent bootstrapping tablet: real 0.012s user 0.002s sys 0.000s
I20260924 13:49:17.730005 25738 tablet_bootstrap.cc:492] T f89b886e7d1c491f80c5d138382d726c P c735360ad0164088af9122a62b4a7b1f: No bootstrap required, opened a new log
I20260924 13:49:17.730091 25738 ts_tablet_manager.cc:1397] T f89b886e7d1c491f80c5d138382d726c P c735360ad0164088af9122a62b4a7b1f: Time spent bootstrapping tablet: real 0.009s user 0.001s sys 0.000s
I20260924 13:49:17.732542 25738 raft_consensus.cc:348] T f89b886e7d1c491f80c5d138382d726c P c735360ad0164088af9122a62b4a7b1f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14d6435e29a641c784cf961d09c28019" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 44457 } } peers { permanent_uuid: "c735360ad0164088af9122a62b4a7b1f" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 45131 } } peers { permanent_uuid: "8a8f26947ec54029bca8c8567da11cf3" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 38515 } }
I20260924 13:49:17.732724 25738 raft_consensus.cc:374] T f89b886e7d1c491f80c5d138382d726c P c735360ad0164088af9122a62b4a7b1f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 13:49:17.732753 25738 raft_consensus.cc:729] T f89b886e7d1c491f80c5d138382d726c P c735360ad0164088af9122a62b4a7b1f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c735360ad0164088af9122a62b4a7b1f, State: Initialized, Role: FOLLOWER
I20260924 13:49:17.732877 25738 consensus_queue.cc:260] T f89b886e7d1c491f80c5d138382d726c P c735360ad0164088af9122a62b4a7b1f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14d6435e29a641c784cf961d09c28019" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 44457 } } peers { permanent_uuid: "c735360ad0164088af9122a62b4a7b1f" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 45131 } } peers { permanent_uuid: "8a8f26947ec54029bca8c8567da11cf3" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 38515 } }
I20260924 13:49:17.733155 25738 ts_tablet_manager.cc:1428] T f89b886e7d1c491f80c5d138382d726c P c735360ad0164088af9122a62b4a7b1f: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260924 13:49:17.736373 25739 raft_consensus.cc:348] T f89b886e7d1c491f80c5d138382d726c P 8a8f26947ec54029bca8c8567da11cf3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14d6435e29a641c784cf961d09c28019" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 44457 } } peers { permanent_uuid: "c735360ad0164088af9122a62b4a7b1f" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 45131 } } peers { permanent_uuid: "8a8f26947ec54029bca8c8567da11cf3" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 38515 } }
I20260924 13:49:17.736581 25739 raft_consensus.cc:374] T f89b886e7d1c491f80c5d138382d726c P 8a8f26947ec54029bca8c8567da11cf3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 13:49:17.736613 25739 raft_consensus.cc:729] T f89b886e7d1c491f80c5d138382d726c P 8a8f26947ec54029bca8c8567da11cf3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a8f26947ec54029bca8c8567da11cf3, State: Initialized, Role: FOLLOWER
I20260924 13:49:17.736726 25739 consensus_queue.cc:260] T f89b886e7d1c491f80c5d138382d726c P 8a8f26947ec54029bca8c8567da11cf3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14d6435e29a641c784cf961d09c28019" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 44457 } } peers { permanent_uuid: "c735360ad0164088af9122a62b4a7b1f" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 45131 } } peers { permanent_uuid: "8a8f26947ec54029bca8c8567da11cf3" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 38515 } }
I20260924 13:49:17.736997 25739 ts_tablet_manager.cc:1428] T f89b886e7d1c491f80c5d138382d726c P 8a8f26947ec54029bca8c8567da11cf3: Time spent starting tablet: real 0.011s user 0.003s sys 0.000s
I20260924 13:49:17.737140 25585 heartbeater.cc:503] Master 127.13.149.254:34083 was elected leader, sending a full tablet report...
I20260924 13:49:17.737510 25740 tablet_bootstrap.cc:492] T f89b886e7d1c491f80c5d138382d726c P 14d6435e29a641c784cf961d09c28019: No bootstrap required, opened a new log
I20260924 13:49:17.737577 25740 ts_tablet_manager.cc:1397] T f89b886e7d1c491f80c5d138382d726c P 14d6435e29a641c784cf961d09c28019: Time spent bootstrapping tablet: real 0.014s user 0.000s sys 0.001s
I20260924 13:49:17.740692 25447 heartbeater.cc:503] Master 127.13.149.254:34083 was elected leader, sending a full tablet report...
I20260924 13:49:17.739480 25740 raft_consensus.cc:348] T f89b886e7d1c491f80c5d138382d726c P 14d6435e29a641c784cf961d09c28019 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14d6435e29a641c784cf961d09c28019" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 44457 } } peers { permanent_uuid: "c735360ad0164088af9122a62b4a7b1f" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 45131 } } peers { permanent_uuid: "8a8f26947ec54029bca8c8567da11cf3" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 38515 } }
I20260924 13:49:17.744716 25740 raft_consensus.cc:374] T f89b886e7d1c491f80c5d138382d726c P 14d6435e29a641c784cf961d09c28019 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 13:49:17.744838 25740 raft_consensus.cc:729] T f89b886e7d1c491f80c5d138382d726c P 14d6435e29a641c784cf961d09c28019 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 14d6435e29a641c784cf961d09c28019, State: Initialized, Role: FOLLOWER
I20260924 13:49:17.745060 25740 consensus_queue.cc:260] T f89b886e7d1c491f80c5d138382d726c P 14d6435e29a641c784cf961d09c28019 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14d6435e29a641c784cf961d09c28019" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 44457 } } peers { permanent_uuid: "c735360ad0164088af9122a62b4a7b1f" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 45131 } } peers { permanent_uuid: "8a8f26947ec54029bca8c8567da11cf3" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 38515 } }
I20260924 13:49:17.745420 25740 ts_tablet_manager.cc:1428] T f89b886e7d1c491f80c5d138382d726c P 14d6435e29a641c784cf961d09c28019: Time spent starting tablet: real 0.008s user 0.000s sys 0.003s
I20260924 13:49:17.745572 25719 heartbeater.cc:503] Master 127.13.149.254:34083 was elected leader, sending a full tablet report...
Sep 24 13:49:17 dist-test-slave-sjl2 krb5kdc[24759](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257757, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.149.254@KRBTEST.COM for HTTP/127.13.149.212@KRBTEST.COM
W20260924 13:49:17.823632 25448 tablet.cc:2369] T f89b886e7d1c491f80c5d138382d726c P c735360ad0164088af9122a62b4a7b1f: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260924 13:49:17.864125 25586 tablet.cc:2369] T f89b886e7d1c491f80c5d138382d726c P 8a8f26947ec54029bca8c8567da11cf3: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260924 13:49:17.890028 25720 tablet.cc:2369] T f89b886e7d1c491f80c5d138382d726c P 14d6435e29a641c784cf961d09c28019: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260924 13:49:17.905501 25744 raft_consensus.cc:482] T f89b886e7d1c491f80c5d138382d726c P c735360ad0164088af9122a62b4a7b1f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 13:49:17.905654 25744 raft_consensus.cc:504] T f89b886e7d1c491f80c5d138382d726c P c735360ad0164088af9122a62b4a7b1f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14d6435e29a641c784cf961d09c28019" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 44457 } } peers { permanent_uuid: "c735360ad0164088af9122a62b4a7b1f" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 45131 } } peers { permanent_uuid: "8a8f26947ec54029bca8c8567da11cf3" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 38515 } }
I20260924 13:49:17.906054 25744 leader_election.cc:288] T f89b886e7d1c491f80c5d138382d726c P c735360ad0164088af9122a62b4a7b1f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 14d6435e29a641c784cf961d09c28019 (127.13.149.195:44457), 8a8f26947ec54029bca8c8567da11cf3 (127.13.149.194:38515)
I20260924 13:49:17.919094 25735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:49:17.906505 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.149.193:52771 (local address 127.13.149.194:38515)
0924 13:49:17.906555 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0924 13:49:17.906557 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:49:17.915585 (+ 9028us) server_negotiation.cc:407] Connection header received
0924 13:49:17.915685 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:49:17.915686 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:49:17.915698 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:49:17.915727 (+ 29us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 13:49:17.915961 (+ 234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:49:17.916631 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:49:17.918145 (+ 1514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:49:17.918710 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:49:17.918760 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:49:17.918770 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0924 13:49:17.918973 (+ 203us) server_negotiation.cc:299] Negotiation successful
0924 13:49:17.919016 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260924 13:49:17.919546 25747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:49:17.906312 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.194:38515 (local address 127.13.149.193:52771)
0924 13:49:17.915530 (+ 9218us) negotiation.cc:107] Waiting for socket to connect
0924 13:49:17.915539 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0924 13:49:17.915608 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:49:17.915792 (+ 184us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:49:17.915793 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:49:17.915806 (+ 13us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 13:49:17.915887 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:49:17.915889 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:49:17.916762 (+ 873us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:49:17.916763 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:49:17.918031 (+ 1268us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:49:17.918036 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:49:17.918829 (+ 793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:49:17.918830 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:49:17.918901 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:49:17.918905 (+ 4us) client_negotiation.cc:769] Sending connection context
0924 13:49:17.919414 (+ 509us) client_negotiation.cc:240] Negotiation successful
0924 13:49:17.919462 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9202,"thread_start_us":93,"threads_started":1}
I20260924 13:49:17.920413 25539 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f89b886e7d1c491f80c5d138382d726c" candidate_uuid: "c735360ad0164088af9122a62b4a7b1f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a8f26947ec54029bca8c8567da11cf3" is_pre_election: true
I20260924 13:49:17.920593 25539 raft_consensus.cc:2454] T f89b886e7d1c491f80c5d138382d726c P 8a8f26947ec54029bca8c8567da11cf3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c735360ad0164088af9122a62b4a7b1f in term 0.
I20260924 13:49:17.920967 25335 leader_election.cc:302] T f89b886e7d1c491f80c5d138382d726c P c735360ad0164088af9122a62b4a7b1f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a8f26947ec54029bca8c8567da11cf3, c735360ad0164088af9122a62b4a7b1f; no voters:
I20260924 13:49:17.921111 25744 raft_consensus.cc:2781] T f89b886e7d1c491f80c5d138382d726c P c735360ad0164088af9122a62b4a7b1f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 13:49:17.921155 25744 raft_consensus.cc:482] T f89b886e7d1c491f80c5d138382d726c P c735360ad0164088af9122a62b4a7b1f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 13:49:17.921175 25744 raft_consensus.cc:3037] T f89b886e7d1c491f80c5d138382d726c P c735360ad0164088af9122a62b4a7b1f [term 0 FOLLOWER]: Advancing to term 1
I20260924 13:49:17.922039 25744 raft_consensus.cc:504] T f89b886e7d1c491f80c5d138382d726c P c735360ad0164088af9122a62b4a7b1f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14d6435e29a641c784cf961d09c28019" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 44457 } } peers { permanent_uuid: "c735360ad0164088af9122a62b4a7b1f" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 45131 } } peers { permanent_uuid: "8a8f26947ec54029bca8c8567da11cf3" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 38515 } }
I20260924 13:49:17.922173 25744 leader_election.cc:288] T f89b886e7d1c491f80c5d138382d726c P c735360ad0164088af9122a62b4a7b1f [CANDIDATE]: Term 1 election: Requested vote from peers 14d6435e29a641c784cf961d09c28019 (127.13.149.195:44457), 8a8f26947ec54029bca8c8567da11cf3 (127.13.149.194:38515)
I20260924 13:49:17.922514 25539 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f89b886e7d1c491f80c5d138382d726c" candidate_uuid: "c735360ad0164088af9122a62b4a7b1f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a8f26947ec54029bca8c8567da11cf3"
I20260924 13:49:17.922596 25539 raft_consensus.cc:3037] T f89b886e7d1c491f80c5d138382d726c P 8a8f26947ec54029bca8c8567da11cf3 [term 0 FOLLOWER]: Advancing to term 1
I20260924 13:49:17.923457 25539 raft_consensus.cc:2454] T f89b886e7d1c491f80c5d138382d726c P 8a8f26947ec54029bca8c8567da11cf3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c735360ad0164088af9122a62b4a7b1f in term 1.
I20260924 13:49:17.923727 25335 leader_election.cc:302] T f89b886e7d1c491f80c5d138382d726c P c735360ad0164088af9122a62b4a7b1f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a8f26947ec54029bca8c8567da11cf3, c735360ad0164088af9122a62b4a7b1f; no voters:
I20260924 13:49:17.923866 25744 raft_consensus.cc:2781] T f89b886e7d1c491f80c5d138382d726c P c735360ad0164088af9122a62b4a7b1f [term 1 FOLLOWER]: Leader election won for term 1
I20260924 13:49:17.924010 25744 raft_consensus.cc:686] T f89b886e7d1c491f80c5d138382d726c P c735360ad0164088af9122a62b4a7b1f [term 1 LEADER]: Becoming Leader. State: Replica: c735360ad0164088af9122a62b4a7b1f, State: Running, Role: LEADER
I20260924 13:49:17.924100 25744 consensus_queue.cc:237] T f89b886e7d1c491f80c5d138382d726c P c735360ad0164088af9122a62b4a7b1f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14d6435e29a641c784cf961d09c28019" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 44457 } } peers { permanent_uuid: "c735360ad0164088af9122a62b4a7b1f" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 45131 } } peers { permanent_uuid: "8a8f26947ec54029bca8c8567da11cf3" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 38515 } }
I20260924 13:49:17.925050 25235 catalog_manager.cc:5495] T f89b886e7d1c491f80c5d138382d726c P c735360ad0164088af9122a62b4a7b1f reported cstate change: term changed from 0 to 1, leader changed from <none> to c735360ad0164088af9122a62b4a7b1f (127.13.149.193). New cstate: current_term: 1 leader_uuid: "c735360ad0164088af9122a62b4a7b1f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14d6435e29a641c784cf961d09c28019" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 44457 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c735360ad0164088af9122a62b4a7b1f" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 45131 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a8f26947ec54029bca8c8567da11cf3" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 38515 } health_report { overall_health: UNKNOWN } } }
I20260924 13:49:17.929774 25737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:49:17.906458 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.149.193:44057 (local address 127.13.149.195:44457)
0924 13:49:17.906555 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0924 13:49:17.906557 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:49:17.915431 (+ 8874us) server_negotiation.cc:407] Connection header received
0924 13:49:17.915503 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:49:17.915504 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:49:17.915519 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:49:17.915595 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 13:49:17.919461 (+ 3866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:49:17.920191 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:49:17.928773 (+ 8582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:49:17.929380 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:49:17.929428 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:49:17.929437 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0924 13:49:17.929645 (+ 208us) server_negotiation.cc:299] Negotiation successful
0924 13:49:17.929689 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260924 13:49:17.930176 25748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:49:17.906227 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.195:44457 (local address 127.13.149.193:44057)
0924 13:49:17.915318 (+ 9091us) negotiation.cc:107] Waiting for socket to connect
0924 13:49:17.915336 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0924 13:49:17.915440 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:49:17.919265 (+ 3825us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:49:17.919267 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:49:17.919270 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 13:49:17.919346 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:49:17.919348 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:49:17.925836 (+ 6488us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:49:17.925837 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:49:17.928650 (+ 2813us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:49:17.928654 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:49:17.929495 (+ 841us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:49:17.929496 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:49:17.929568 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:49:17.929573 (+ 5us) client_negotiation.cc:769] Sending connection context
0924 13:49:17.930025 (+ 452us) client_negotiation.cc:240] Negotiation successful
0924 13:49:17.930076 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9060,"thread_start_us":83,"threads_started":1}
I20260924 13:49:17.930636 25673 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f89b886e7d1c491f80c5d138382d726c" candidate_uuid: "c735360ad0164088af9122a62b4a7b1f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "14d6435e29a641c784cf961d09c28019" is_pre_election: true
I20260924 13:49:17.930804 25673 raft_consensus.cc:2454] T f89b886e7d1c491f80c5d138382d726c P 14d6435e29a641c784cf961d09c28019 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c735360ad0164088af9122a62b4a7b1f in term 0.
I20260924 13:49:17.930927 25672 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f89b886e7d1c491f80c5d138382d726c" candidate_uuid: "c735360ad0164088af9122a62b4a7b1f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "14d6435e29a641c784cf961d09c28019"
I20260924 13:49:17.930990 25672 raft_consensus.cc:3037] T f89b886e7d1c491f80c5d138382d726c P 14d6435e29a641c784cf961d09c28019 [term 0 FOLLOWER]: Advancing to term 1
I20260924 13:49:17.931892 25672 raft_consensus.cc:2454] T f89b886e7d1c491f80c5d138382d726c P 14d6435e29a641c784cf961d09c28019 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c735360ad0164088af9122a62b4a7b1f in term 1.
I20260924 13:49:17.934790 25734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:49:17.931500 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60452 (local address 127.13.149.193:45131)
0924 13:49:17.931534 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0924 13:49:17.931535 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0924 13:49:17.931541 (+ 6us) server_negotiation.cc:407] Connection header received
0924 13:49:17.931569 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:49:17.931570 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:49:17.931587 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:49:17.931639 (+ 52us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 13:49:17.931822 (+ 183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:49:17.932545 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:49:17.933665 (+ 1120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:49:17.933975 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:49:17.934033 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:49:17.934255 (+ 222us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 13:49:17.934380 (+ 125us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 13:49:17.934451 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0924 13:49:17.934672 (+ 221us) server_negotiation.cc:299] Negotiation successful
0924 13:49:17.934692 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260924 13:49:17.936797 25539 raft_consensus.cc:1261] T f89b886e7d1c491f80c5d138382d726c P 8a8f26947ec54029bca8c8567da11cf3 [term 1 FOLLOWER]: Refusing update from remote peer c735360ad0164088af9122a62b4a7b1f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 13:49:17.936949 25672 raft_consensus.cc:1261] T f89b886e7d1c491f80c5d138382d726c P 14d6435e29a641c784cf961d09c28019 [term 1 FOLLOWER]: Refusing update from remote peer c735360ad0164088af9122a62b4a7b1f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 13:49:17.937232 25744 consensus_queue.cc:1035] T f89b886e7d1c491f80c5d138382d726c P c735360ad0164088af9122a62b4a7b1f [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a8f26947ec54029bca8c8567da11cf3" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 38515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 13:49:17.937383 25744 consensus_queue.cc:1035] T f89b886e7d1c491f80c5d138382d726c P c735360ad0164088af9122a62b4a7b1f [LEADER]: Connected to new peer: Peer: permanent_uuid: "14d6435e29a641c784cf961d09c28019" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 44457 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 13:49:17.939963 25752 mvcc.cc:205] Tried to move back new op lower bound from 7332895776506531840 to 7332895776457588736. Current Snapshot: MvccSnapshot[applied={T|T < 7332895776506531840}]
I20260924 13:49:17.940946 25754 mvcc.cc:205] Tried to move back new op lower bound from 7332895776506531840 to 7332895776457588736. Current Snapshot: MvccSnapshot[applied={T|T < 7332895776506531840}]
I20260924 13:49:17.944901 25234 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52786:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 13:49:17.945022 25234 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52786:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 13:49:17.945881 25234 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5c231d9794ba4625856b262d5d874e35: Sending DeleteTablet for 3 replicas of tablet f89b886e7d1c491f80c5d138382d726c
I20260924 13:49:17.946286 25381 tablet_service.cc:1514] Processing DeleteTablet for tablet f89b886e7d1c491f80c5d138382d726c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 13:49:17 UTC) from {username='kudu', principal='kudu/127.13.149.254@KRBTEST.COM'} at 127.0.0.1:60446
I20260924 13:49:17.946501 25519 tablet_service.cc:1514] Processing DeleteTablet for tablet f89b886e7d1c491f80c5d138382d726c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 13:49:17 UTC) from {username='kudu', principal='kudu/127.13.149.254@KRBTEST.COM'} at 127.0.0.1:53028
I20260924 13:49:17.946883 25653 tablet_service.cc:1514] Processing DeleteTablet for tablet f89b886e7d1c491f80c5d138382d726c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 13:49:17 UTC) from {username='kudu', principal='kudu/127.13.149.254@KRBTEST.COM'} at 127.0.0.1:43596
I20260924 13:49:17.947611 13911 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJN9WRF/build/release/bin/kudu with pid 25316
I20260924 13:49:17.949045 25764 tablet_replica.cc:330] T f89b886e7d1c491f80c5d138382d726c P 14d6435e29a641c784cf961d09c28019: stopping tablet replica
I20260924 13:49:17.949216 25764 raft_consensus.cc:2229] T f89b886e7d1c491f80c5d138382d726c P 14d6435e29a641c784cf961d09c28019 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 13:49:17.949409 25764 raft_consensus.cc:2258] T f89b886e7d1c491f80c5d138382d726c P 14d6435e29a641c784cf961d09c28019 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 13:49:17.949982 25763 tablet_replica.cc:330] T f89b886e7d1c491f80c5d138382d726c P 8a8f26947ec54029bca8c8567da11cf3: stopping tablet replica
I20260924 13:49:17.950157 25763 raft_consensus.cc:2229] T f89b886e7d1c491f80c5d138382d726c P 8a8f26947ec54029bca8c8567da11cf3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 13:49:17.950323 25763 raft_consensus.cc:2258] T f89b886e7d1c491f80c5d138382d726c P 8a8f26947ec54029bca8c8567da11cf3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 13:49:17.950894 25763 ts_tablet_manager.cc:1905] T f89b886e7d1c491f80c5d138382d726c P 8a8f26947ec54029bca8c8567da11cf3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 13:49:17.952937 25764 ts_tablet_manager.cc:1905] T f89b886e7d1c491f80c5d138382d726c P 14d6435e29a641c784cf961d09c28019: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260924 13:49:17.953259 25219 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.13.149.193:45131 (error 108)
I20260924 13:49:17.953820 13911 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJN9WRF/build/release/bin/kudu with pid 25455
I20260924 13:49:17.955127 25764 ts_tablet_manager.cc:1918] T f89b886e7d1c491f80c5d138382d726c P 14d6435e29a641c784cf961d09c28019: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 13:49:17.955204 25764 log.cc:1196] T f89b886e7d1c491f80c5d138382d726c P 14d6435e29a641c784cf961d09c28019: Deleting WAL directory at /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790257629041645-13911-0/minicluster-data/ts-2/wal/wals/f89b886e7d1c491f80c5d138382d726c
I20260924 13:49:17.955755 25764 ts_tablet_manager.cc:1939] T f89b886e7d1c491f80c5d138382d726c P 14d6435e29a641c784cf961d09c28019: Deleting consensus metadata
I20260924 13:49:17.956296 25733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:49:17.956128 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.193:45131 (local address 127.0.0.1:60456)
0924 13:49:17.956182 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0924 13:49:17.956234 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.149.193:45131: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260924 13:49:17.956442 25219 catalog_manager.cc:4584] TS c735360ad0164088af9122a62b4a7b1f (127.13.149.193:45131): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet f89b886e7d1c491f80c5d138382d726c: Network error: Client connection negotiation failed: client connection to 127.13.149.193:45131: connect: Connection refused (error 111)
I20260924 13:49:17.956952 25221 catalog_manager.cc:4841] TS 14d6435e29a641c784cf961d09c28019 (127.13.149.195:44457): tablet f89b886e7d1c491f80c5d138382d726c (table test-table [id=9557c0b6bbfa4e5fac47b9ebff54660d]) successfully deleted
I20260924 13:49:17.959412 25733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:49:17.959270 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.194:38515 (local address 127.0.0.1:53038)
0924 13:49:17.959342 (+ 72us) negotiation.cc:107] Waiting for socket to connect
0924 13:49:17.959377 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.149.194:38515: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260924 13:49:17.959499 13911 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJN9WRF/build/release/bin/kudu with pid 25590
I20260924 13:49:17.963737 13911 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJN9WRF/build/release/bin/kudu with pid 25203
I20260924 13:49:17.968240 13911 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260924 13:49:18.302786 13911 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260924 13:49:18.302910 13911 mini_ranger.cc:66] Stopping Ranger...
I20260924 13:49:18.637416 13911 mini_ranger.cc:68] Stopped Ranger
2026-09-24 13:49:18.637 UTC [24789] LOG: received smart shutdown request
2026-09-24 13:49:18.641 UTC [24789] LOG: background worker "logical replication launcher" (PID 24798) exited with exit code 1
2026-09-24 13:49:18.641 UTC [24793] LOG: shutting down
2026-09-24 13:49:18.809 UTC [24789] LOG: database system is shut down
2026-09-24T13:49:18Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (45901 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790257629041645-13911-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 13:49:18 dist-test-slave-sjl2 krb5kdc[25774](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 13:49:18 dist-test-slave-sjl2 krb5kdc[25774](info): set up 2 sockets
Sep 24 13:49:18 dist-test-slave-sjl2 krb5kdc[25774](info): commencing operation
krb5kdc: starting...
W20260924 13:49:19.841845 13911 mini_kdc.cc:121] Time spent starting KDC: real 0.933s user 0.002s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790257629041645-13911-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790257629041645-13911-0/krb5kdc/test-user.keytab.
Sep 24 13:49:19 dist-test-slave-sjl2 krb5kdc[25774](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257759, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T13:49:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T13:49:19Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.13.149.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.13.149.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.13.149.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790257629041645-13911-0/krb5kdc/rangeradmin_127.13.149.212@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.13.149.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790257629041645-13911-0/krb5kdc/rangeradmin_127.13.149.212@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.13.149.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.13.149.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.13.149.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790257629041645-13911-0/krb5kdc/rangerlookup_127.13.149.212@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.13.149.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790257629041645-13911-0/krb5kdc/rangerlookup_127.13.149.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.13.149.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.13.149.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.13.149.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790257629041645-13911-0/krb5kdc/HTTP_127.13.149.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.13.149.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790257629041645-13911-0/krb5kdc/HTTP_127.13.149.212@KRBTEST.COM.keytab.
I20260924 13:49:20.005740 13911 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790257629041645-13911-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskJN9WRF/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790257629041645-13911-0/minicluster-data/postgres -l logfile start
2026-09-24 13:49:22.662 UTC [25804] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-09-24 13:49:22.662 UTC [25804] LOG: listening on IPv4 address "127.13.149.212", port 32845
2026-09-24 13:49:22.666 UTC [25804] LOG: listening on Unix socket "/tmp/.s.PGSQL.32845"
2026-09-24 13:49:22.686 UTC [25807] LOG: database system was shut down at 2026-09-24 13:49:21 UTC
2026-09-24 13:49:22.689 UTC [25804] LOG: database system is ready to accept connections
I20260924 13:49:23.561655 13911 mini_postgres.cc:96] Postgres bound to 32845
2026-09-24 13:49:23.567 UTC [25815] FATAL: database "slave" does not exist
127.13.149.212:32845 - accepting connections
I20260924 13:49:23.567935 13911 mini_ranger.cc:161] Starting Ranger...
I20260924 13:49:23.582370 13911 mini_ranger.cc:84] Created miniranger Postgres user
I20260924 13:49:24.123735 13911 mini_ranger.cc:87] Created ranger Postgres database
I20260924 13:49:24.123835 13911 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790257629041645-13911-0/minicluster-data/ranger-admin
2026-09-24 13:49:24,399 [I] DB FLAVOR :POSTGRES
2026-09-24 13:49:24,399 [I] --------- Verifying Ranger DB connection ---------
2026-09-24 13:49:24,399 [I] Checking connection..
2026-09-24 13:49:24,399 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJN9WRF/build/release/bin/postgresql.jar:/tmp/dist-test-taskJN9WRF/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.149.212:32845/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-09-24 13:49:24,659 [I] Checking connection passed.
2026-09-24 13:49:24,659 [I] --------- Verifying version history table ---------
2026-09-24 13:49:24,659 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJN9WRF/build/release/bin/postgresql.jar:/tmp/dist-test-taskJN9WRF/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.149.212:32845/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-24 13:49:24,897 [I] Table x_db_version_h does not exist in database ranger
2026-09-24 13:49:24,898 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJN9WRF/build/release/bin/postgresql.jar:/tmp/dist-test-taskJN9WRF/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.149.212:32845/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-24 13:49:25,161 [I] Table x_db_version_h does not exist in database ranger
2026-09-24 13:49:25,161 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-09-24 13:49:25,161 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJN9WRF/build/release/bin/postgresql.jar:/tmp/dist-test-taskJN9WRF/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.149.212:32845/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskJN9WRF/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-09-24 13:49:25.411 UTC [25879] WARNING: there is no transaction in progress
2026-09-24 13:49:25,418 [I] create_dbversion_catalog.sql file imported successfully
2026-09-24 13:49:25,418 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJN9WRF/build/release/bin/postgresql.jar:/tmp/dist-test-taskJN9WRF/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.149.212:32845/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-24 13:49:25,665 [I] Table x_db_version_h already exists in database 'ranger'
2026-09-24 13:49:25,665 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-24 13:49:25,666 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJN9WRF/build/release/bin/postgresql.jar:/tmp/dist-test-taskJN9WRF/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.149.212:32845/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-09-24 13:49:25,900 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJN9WRF/build/release/bin/postgresql.jar:/tmp/dist-test-taskJN9WRF/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.149.212:32845/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-09-24 13:49:26,160 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJN9WRF/build/release/bin/postgresql.jar:/tmp/dist-test-taskJN9WRF/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.149.212:32845/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-sjl2.c.gcp-upstream.internal','N') ;"
2026-09-24 13:49:26,414 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJN9WRF/build/release/bin/postgresql.jar:/tmp/dist-test-taskJN9WRF/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.149.212:32845/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-24 13:49:26,658 [I] Table x_portal_user does not exist in database ranger
2026-09-24 13:49:26,659 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJN9WRF/build/release/bin/postgresql.jar:/tmp/dist-test-taskJN9WRF/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.149.212:32845/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-24 13:49:26,889 [I] Table x_policy_ref_group does not exist in database ranger
2026-09-24 13:49:26,890 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJN9WRF/build/release/bin/postgresql.jar:/tmp/dist-test-taskJN9WRF/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.149.212:32845/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-24 13:49:27,134 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJN9WRF/build/release/bin/postgresql.jar:/tmp/dist-test-taskJN9WRF/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.149.212:32845/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-24 13:49:27,374 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-09-24 13:49:27,374 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJN9WRF/build/release/bin/postgresql.jar:/tmp/dist-test-taskJN9WRF/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.149.212:32845/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskJN9WRF/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-09-24 13:49:28.359 UTC [25987] WARNING: there is no transaction in progress
2026-09-24 13:49:28.376 UTC [25987] WARNING: there is no transaction in progress
2026-09-24 13:49:28.394 UTC [25987] WARNING: there is no transaction in progress
2026-09-24 13:49:28.410 UTC [25987] WARNING: there is no transaction in progress
2026-09-24 13:49:28.426 UTC [25987] WARNING: there is no transaction in progress
2026-09-24 13:49:28.449 UTC [25987] WARNING: there is no transaction in progress
2026-09-24 13:49:28.509 UTC [25987] WARNING: there is no transaction in progress
2026-09-24 13:49:28.517 UTC [25987] WARNING: there is no transaction in progress
2026-09-24 13:49:28.532 UTC [25987] WARNING: there is no transaction in progress
2026-09-24 13:49:28.542 UTC [25987] WARNING: there is no transaction in progress
2026-09-24 13:49:28.552 UTC [25987] WARNING: there is no transaction in progress
2026-09-24 13:49:28.566 UTC [25987] WARNING: there is no transaction in progress
2026-09-24 13:49:28.577 UTC [25987] WARNING: there is no transaction in progress
2026-09-24 13:49:28.584 UTC [25987] WARNING: there is no transaction in progress
2026-09-24 13:49:28.591 UTC [25987] WARNING: there is no transaction in progress
2026-09-24 13:49:28.598 UTC [25987] WARNING: there is no transaction in progress
2026-09-24 13:49:29,296 [I] ranger_core_db_postgres.sql file imported successfully
2026-09-24 13:49:29,297 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJN9WRF/build/release/bin/postgresql.jar:/tmp/dist-test-taskJN9WRF/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.149.212:32845/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-09-24 13:49:29,543 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-09-24 13:49:29,543 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJN9WRF/build/release/bin/postgresql.jar:/tmp/dist-test-taskJN9WRF/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.149.212:32845/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-24 13:49:29,788 [I] Table x_portal_user already exists in database 'ranger'
2026-09-24 13:49:29,788 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJN9WRF/build/release/bin/postgresql.jar:/tmp/dist-test-taskJN9WRF/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.149.212:32845/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-24 13:49:30,050 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-09-24 13:49:30,050 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJN9WRF/build/release/bin/postgresql.jar:/tmp/dist-test-taskJN9WRF/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.149.212:32845/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-24 13:49:30,308 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJN9WRF/build/release/bin/postgresql.jar:/tmp/dist-test-taskJN9WRF/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.149.212:32845/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-24 13:49:30,556 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJN9WRF/build/release/bin/postgresql.jar:/tmp/dist-test-taskJN9WRF/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.149.212:32845/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-sjl2.c.gcp-upstream.internal';"
2026-09-24 13:49:30,807 [I] CORE_DB_SCHEMA import status has been updated
2026-09-24 13:49:30,807 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJN9WRF/build/release/bin/postgresql.jar:/tmp/dist-test-taskJN9WRF/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.149.212:32845/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-09-24 13:49:31,068 [I] DB_PATCHES have already been applied
I20260924 13:49:31.074033 13911 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790257629041645-13911-0/minicluster-data/ranger-admin:/tmp/dist-test-taskJN9WRF/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskJN9WRF/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskJN9WRF/build/release/bin/postgresql.jar:/tmp/dist-test-taskJN9WRF/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260924 13:49:31.074115 13911 mini_ranger.cc:193] Using host: 127.13.149.212
I20260924 13:49:31.077092 13911 mini_ranger.cc:227] Ranger admin URL: http://127.13.149.212:60259
Sep 24, 2026 1:49:31 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 24, 2026 1:49:31 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 24, 2026 1:49:31 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790257629041645-13911-0/minicluster-data/ranger-admin/ews/webapp
Sep 24, 2026 1:49:31 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790257629041645-13911-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Sep 24, 2026 1:49:31 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790257629041645-13911-0/minicluster-data/ranger-admin/ews/webapp] .....
Sep 24, 2026 1:49:31 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790257629041645-13911-0/minicluster-data/ranger-admin/ews/webapp].
Sep 24, 2026 1:49:31 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260924 13:49:32.086427 13911 mini_ranger.cc:160] Time spent starting Ranger: real 8.518s user 0.000s sys 0.008s
Sep 24, 2026 1:49:51 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Sep 24, 2026 1:49:51 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.UserREST
Sep 24, 2026 1:49:51 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Sep 24, 2026 1:49:51 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Sep 24, 2026 1:49:51 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Sep 24, 2026 1:49:51 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Sep 24, 2026 1:49:51 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Sep 24, 2026 1:49:51 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Sep 24, 2026 1:49:51 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Sep 24, 2026 1:49:51 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Sep 24, 2026 1:49:51 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Sep 24, 2026 1:49:51 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Sep 24, 2026 1:49:51 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Sep 24, 2026 1:49:51 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Sep 24, 2026 1:49:51 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Sep 24, 2026 1:49:51 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Sep 24, 2026 1:49:51 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Sep 24, 2026 1:49:52 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260924 13:49:52.605839 13911 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.13.149.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.13.149.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.13.149.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790257629041645-13911-0/krb5kdc/rangerkms_127.13.149.212@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.13.149.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790257629041645-13911-0/krb5kdc/rangerkms_127.13.149.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.13.149.212@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.13.149.212@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.13.149.212@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790257629041645-13911-0/krb5kdc/HTTP_127.13.149.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.13.149.212@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790257629041645-13911-0/krb5kdc/HTTP_127.13.149.212@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Sep 24 13:49:52 dist-test-slave-sjl2 krb5kdc[25774](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257792, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260924 13:49:52.686498 13911 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260924 13:49:52.695477 13911 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260924 13:49:53.597474 13911 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260924 13:49:53.597555 13911 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790257629041645-13911-0/minicluster-data/ranger-kms
I20260924 13:49:53.597571 13911 mini_ranger_kms.cc:227] Using postgres at 127.13.149.212:32845
2026-09-24 13:49:53,753 [I] DB FLAVOR :POSTGRES
2026-09-24 13:49:53,753 [I] --------- Verifying Ranger DB connection ---------
2026-09-24 13:49:53,753 [I] Checking connection
2026-09-24 13:49:53,753 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJN9WRF/build/release/bin/postgresql.jar:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790257629041645-13911-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.149.212:32845/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-09-24 13:49:53,985 [I] connection success
2026-09-24 13:49:53,985 [I] --------- Verifying Ranger DB tables ---------
2026-09-24 13:49:53,985 [I] Verifying table ranger_masterkey in database rangerkms
2026-09-24 13:49:53,986 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJN9WRF/build/release/bin/postgresql.jar:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790257629041645-13911-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.149.212:32845/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-09-24 13:49:54,218 [I] Table ranger_masterkey does not exist in database rangerkms
2026-09-24 13:49:54,218 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-24 13:49:54,218 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-09-24 13:49:54,218 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJN9WRF/build/release/bin/postgresql.jar:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790257629041645-13911-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.149.212:32845/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790257629041645-13911-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-09-24 13:49:54,473 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260924 13:49:54.892159 13911 mini_ranger_kms.cc:310] Created kms service
I20260924 13:49:55.073899 13911 mini_ranger_kms.cc:326] Created kudu user
I20260924 13:49:55.119176 13911 mini_ranger_kms.cc:343] Created rangerkms user
Sep 24 13:49:55 dist-test-slave-sjl2 krb5kdc[25774](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257792, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.13.149.212@KRBTEST.COM
I20260924 13:49:55.513548 13911 mini_ranger_kms.cc:384] Added ranger policy
I20260924 13:49:55.513767 13911 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790257629041645-13911-0/minicluster-data/ranger-kms:/tmp/dist-test-taskJN9WRF/build/release/bin/postgresql.jar:/tmp/dist-test-taskJN9WRF/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskJN9WRF/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260924 13:49:55.513787 13911 mini_ranger_kms.cc:240] Using host: 127.13.149.212
I20260924 13:49:55.517323 13911 mini_ranger_kms.cc:276] Ranger KMS PID: 26167
I20260924 13:49:55.517429 13911 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.13.149.212:55543
Sep 24, 2026 1:49:56 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 24, 2026 1:49:56 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 24, 2026 1:49:56 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790257629041645-13911-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Sep 24, 2026 1:49:56 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790257629041645-13911-0/minicluster-data/ranger-kms/ews/webapp] .....
Sep 24, 2026 1:49:57 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790257629041645-13911-0/minicluster-data/ranger-kms/ews/webapp].
Sep 24, 2026 1:49:57 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260924 13:49:57.565249 13911 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 4.879s user 0.001s sys 0.137s
I20260924 13:49:57.565357 13911 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260924 13:49:57.565404 13911 mini_ranger_kms.cc:401] 127.13.149.212:55543/kms/v1/keys
Sep 24 13:49:57 dist-test-slave-sjl2 krb5kdc[25774](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790257797, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.13.149.212@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Sep 24 13:49:58 dist-test-slave-sjl2 krb5kdc[25774](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257797, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.13.149.212@KRBTEST.COM for HTTP/127.13.149.212@KRBTEST.COM
Sep 24 13:49:58 dist-test-slave-sjl2 krb5kdc[25774](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257797, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.13.149.212@KRBTEST.COM for HTTP/127.13.149.212@KRBTEST.COM
Sep 24 13:50:00 dist-test-slave-sjl2 krb5kdc[25774](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257800, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.13.149.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.149.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.149.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.254.keytab.
Entry for principal kudu/127.13.149.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.254.keytab.
I20260924 13:50:01.030341 13911 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJN9WRF/build/release/bin/kudu
/tmp/dist-test-taskJN9WRF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790257629041645-13911-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790257629041645-13911-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790257629041645-13911-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790257629041645-13911-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.149.254:34283
--webserver_interface=127.13.149.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.254.keytab
--principal=kudu/127.13.149.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.149.212:38701
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.149.254:34283
--ranger_config_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790257629041645-13911-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.13.149.212:55543/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790257629041645-13911-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790257629041645-13911-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790257629041645-13911-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 13:50:01.112385 26218 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 13:50:01.112620 26218 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 13:50:01.112653 26218 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 13:50:01.113924 26218 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 13:50:01.113973 26218 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 13:50:01.113993 26218 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 13:50:01.114008 26218 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 13:50:01.114024 26218 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 13:50:01.115638 26218 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.149.212:38701
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790257629041645-13911-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790257629041645-13911-0/minicluster-data/master-0/wal
--ranger_kms_url=127.13.149.212:55543/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.13.149.254:34283
--ranger_config_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790257629041645-13911-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.254.keytab
--principal=kudu/127.13.149.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.149.254:34283
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790257629041645-13911-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.149.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26218
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790257629041645-13911-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 12:29:37 UTC on ccycloud.quasar-iqzbxn.root.comops.site
build id 4670106
I20260924 13:50:01.115890 26218 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 13:50:01.116106 26218 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 13:50:01.121567 26223 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 13:50:01.121649 26218 server_base.cc:1034] running on GCE node
W20260924 13:50:01.121729 26224 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 13:50:01.121568 26226 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 13:50:01.122093 26218 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 13:50:01.122408 26218 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 13:50:01.123577 26218 hybrid_clock.cc:648] HybridClock initialized: now 1790257801123550 us; error 45 us; skew 500 ppm
Sep 24 13:50:01 dist-test-slave-sjl2 krb5kdc[25774](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257801, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.149.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 13:50:01.130035 26218 init.cc:377] Logged in from keytab as kudu/127.13.149.254@KRBTEST.COM (short username kudu)
I20260924 13:50:01.130527 26218 webserver.cc:460] Webserver started at http://127.13.149.254:42081/ using document root <none> and password file <none>
I20260924 13:50:01.130748 26218 fs_manager.cc:359] Metadata directory not provided
I20260924 13:50:01.130786 26218 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 13:50:01.130913 26218 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 24 13:50:01 dist-test-slave-sjl2 krb5kdc[25774](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257801, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.149.254@KRBTEST.COM for HTTP/127.13.149.212@KRBTEST.COM
I20260924 13:50:01.194054 26218 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790257629041645-13911-0/minicluster-data/master-0/data/instance:
uuid: "9bb72cc199554d53af2e97d01120dbd8"
format_stamp: "Formatted at 2026-09-24 13:50:01 on dist-test-slave-sjl2"
server_key: "e075b27eb8dc25a7f89cad91fd00701e"
server_key_iv: "e220485a69ecf78a84757a523475fce2"
server_key_version: "kuduclusterkey@0"
I20260924 13:50:01.194509 26218 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790257629041645-13911-0/minicluster-data/master-0/wal/instance:
uuid: "9bb72cc199554d53af2e97d01120dbd8"
format_stamp: "Formatted at 2026-09-24 13:50:01 on dist-test-slave-sjl2"
server_key: "e075b27eb8dc25a7f89cad91fd00701e"
server_key_iv: "e220485a69ecf78a84757a523475fce2"
server_key_version: "kuduclusterkey@0"
I20260924 13:50:01.196211 26218 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.001s sys 0.001s
I20260924 13:50:01.211694 26233 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 13:50:01.211870 26218 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260924 13:50:01.211962 26218 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790257629041645-13911-0/minicluster-data/master-0/data,/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790257629041645-13911-0/minicluster-data/master-0/wal
uuid: "9bb72cc199554d53af2e97d01120dbd8"
format_stamp: "Formatted at 2026-09-24 13:50:01 on dist-test-slave-sjl2"
server_key: "e075b27eb8dc25a7f89cad91fd00701e"
server_key_iv: "e220485a69ecf78a84757a523475fce2"
server_key_version: "kuduclusterkey@0"
I20260924 13:50:01.212059 26218 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790257629041645-13911-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790257629041645-13911-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790257629041645-13911-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 13:50:01.234403 26218 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 13:50:01.236603 26218 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 13:50:01.236757 26218 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 13:50:01.241595 26218 rpc_server.cc:307] RPC server started. Bound to: 127.13.149.254:34283
I20260924 13:50:01.241644 26285 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.149.254:34283 every 8 connection(s)
I20260924 13:50:01.242053 26218 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790257629041645-13911-0/minicluster-data/master-0/data/info.pb
I20260924 13:50:01.242813 26286 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 13:50:01.245285 26286 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9bb72cc199554d53af2e97d01120dbd8: Bootstrap starting.
I20260924 13:50:01.245900 26286 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9bb72cc199554d53af2e97d01120dbd8: Neither blocks nor log segments found. Creating new log.
I20260924 13:50:01.246184 26286 log.cc:824] T 00000000000000000000000000000000 P 9bb72cc199554d53af2e97d01120dbd8: Log is configured to *not* fsync() on all Append() calls
I20260924 13:50:01.246863 26286 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9bb72cc199554d53af2e97d01120dbd8: No bootstrap required, opened a new log
I20260924 13:50:01.247169 13911 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJN9WRF/build/release/bin/kudu as pid 26218
I20260924 13:50:01.247280 13911 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790257629041645-13911-0/minicluster-data/master-0/wal/instance
I20260924 13:50:01.247471 13911 external_mini_cluster.cc:1428] Setting key ca5f985492f60f8dd2b687bbd72a5a34
I20260924 13:50:01.249732 26286 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9bb72cc199554d53af2e97d01120dbd8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9bb72cc199554d53af2e97d01120dbd8" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 34283 } }
I20260924 13:50:01.250022 26286 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9bb72cc199554d53af2e97d01120dbd8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 13:50:01.250222 26286 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9bb72cc199554d53af2e97d01120dbd8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9bb72cc199554d53af2e97d01120dbd8, State: Initialized, Role: FOLLOWER
I20260924 13:50:01.250427 26286 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9bb72cc199554d53af2e97d01120dbd8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9bb72cc199554d53af2e97d01120dbd8" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 34283 } }
I20260924 13:50:01.250514 26286 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9bb72cc199554d53af2e97d01120dbd8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 13:50:01.250547 26286 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9bb72cc199554d53af2e97d01120dbd8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 13:50:01.250607 26286 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9bb72cc199554d53af2e97d01120dbd8 [term 0 FOLLOWER]: Advancing to term 1
I20260924 13:50:01.251252 26286 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9bb72cc199554d53af2e97d01120dbd8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9bb72cc199554d53af2e97d01120dbd8" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 34283 } }
I20260924 13:50:01.251358 26286 leader_election.cc:302] T 00000000000000000000000000000000 P 9bb72cc199554d53af2e97d01120dbd8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9bb72cc199554d53af2e97d01120dbd8; no voters:
I20260924 13:50:01.251545 26286 leader_election.cc:288] T 00000000000000000000000000000000 P 9bb72cc199554d53af2e97d01120dbd8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 13:50:01.251587 26291 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9bb72cc199554d53af2e97d01120dbd8 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 13:50:01.251780 26291 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9bb72cc199554d53af2e97d01120dbd8 [term 1 LEADER]: Becoming Leader. State: Replica: 9bb72cc199554d53af2e97d01120dbd8, State: Running, Role: LEADER
I20260924 13:50:01.251900 26286 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9bb72cc199554d53af2e97d01120dbd8 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 13:50:01.251904 26291 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9bb72cc199554d53af2e97d01120dbd8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9bb72cc199554d53af2e97d01120dbd8" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 34283 } }
I20260924 13:50:01.252319 26291 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9bb72cc199554d53af2e97d01120dbd8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9bb72cc199554d53af2e97d01120dbd8. Latest consensus state: current_term: 1 leader_uuid: "9bb72cc199554d53af2e97d01120dbd8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9bb72cc199554d53af2e97d01120dbd8" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 34283 } } }
I20260924 13:50:01.252466 26291 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9bb72cc199554d53af2e97d01120dbd8 [sys.catalog]: This master's current role is: LEADER
I20260924 13:50:01.252655 26293 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9bb72cc199554d53af2e97d01120dbd8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9bb72cc199554d53af2e97d01120dbd8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9bb72cc199554d53af2e97d01120dbd8" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 34283 } } }
I20260924 13:50:01.252780 26293 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9bb72cc199554d53af2e97d01120dbd8 [sys.catalog]: This master's current role is: LEADER
I20260924 13:50:01.253404 26286 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790257629041645-13911-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260924 13:50:01.255693 26300 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
Sep 24 13:50:01 dist-test-slave-sjl2 krb5kdc[25774](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257800, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.149.254@KRBTEST.COM
I20260924 13:50:01.257930 26300 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 13:50:01.262490 26290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:50:01.248034 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49814 (local address 127.13.149.254:34283)
0924 13:50:01.248242 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0924 13:50:01.248253 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0924 13:50:01.248286 (+ 33us) server_negotiation.cc:407] Connection header received
0924 13:50:01.249687 (+ 1401us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:50:01.249698 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:50:01.249772 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:50:01.249945 (+ 173us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:50:01.250744 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:50:01.252045 (+ 1301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:50:01.253012 (+ 967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:50:01.253641 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:50:01.253751 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:50:01.257312 (+ 3561us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:50:01.257348 (+ 36us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:50:01.257348 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 13:50:01.257370 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:50:01.260051 (+ 2681us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:50:01.260496 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:50:01.260501 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:50:01.260502 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:50:01.260631 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:50:01.260829 (+ 198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:50:01.260830 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:50:01.260831 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:50:01.261243 (+ 412us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:50:01.261550 (+ 307us) server_negotiation.cc:1035] Waiting for connection context
0924 13:50:01.262091 (+ 541us) server_negotiation.cc:299] Negotiation successful
0924 13:50:01.262247 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":150,"thread_start_us":92,"threads_started":1}
I20260924 13:50:01.265239 26300 catalog_manager.cc:1347] Generated new cluster ID: ba7fab592cd54f67ab6466657dfe587d
I20260924 13:50:01.265314 26300 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 13:50:01.281224 26300 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 13:50:01.282095 26300 catalog_manager.cc:1504] Loading token signing keys...
I20260924 13:50:01.290058 26300 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9bb72cc199554d53af2e97d01120dbd8: Generated new TSK 0
I20260924 13:50:01.290423 26300 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.13.149.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.149.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.149.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.193.keytab.
Entry for principal kudu/127.13.149.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.193.keytab.
I20260924 13:50:02.044742 13911 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJN9WRF/build/release/bin/kudu
/tmp/dist-test-taskJN9WRF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790257629041645-13911-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790257629041645-13911-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790257629041645-13911-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790257629041645-13911-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.13.149.212:55543/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.149.193:0
--local_ip_for_outbound_sockets=127.13.149.193
--webserver_interface=127.13.149.193
--webserver_port=0
--tserver_master_addrs=127.13.149.254:34283
--keytab_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.193.keytab
--principal=kudu/127.13.149.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.149.212:38701
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790257629041645-13911-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790257629041645-13911-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790257629041645-13911-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 13:50:02.160267 26331 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 13:50:02.160490 26331 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 13:50:02.160523 26331 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 13:50:02.161895 26331 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 13:50:02.161952 26331 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 13:50:02.161993 26331 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.149.193
I20260924 13:50:02.163951 26331 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.149.212:38701
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790257629041645-13911-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790257629041645-13911-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.13.149.212:55543/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.193.keytab
--principal=kudu/127.13.149.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.149.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790257629041645-13911-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.149.193
--webserver_port=0
--tserver_master_addrs=127.13.149.254:34283
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26331
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.149.193
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790257629041645-13911-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 12:29:37 UTC on ccycloud.quasar-iqzbxn.root.comops.site
build id 4670106
I20260924 13:50:02.164394 26331 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 13:50:02.164688 26331 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 13:50:02.171270 26336 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 13:50:02.171341 26339 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 13:50:02.171640 26337 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 13:50:02.171839 26331 server_base.cc:1034] running on GCE node
I20260924 13:50:02.172046 26331 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 13:50:02.172325 26331 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 13:50:02.179807 26331 hybrid_clock.cc:648] HybridClock initialized: now 1790257802179770 us; error 61 us; skew 500 ppm
Sep 24 13:50:02 dist-test-slave-sjl2 krb5kdc[25774](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257802, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.149.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 13:50:02.183385 26331 init.cc:377] Logged in from keytab as kudu/127.13.149.193@KRBTEST.COM (short username kudu)
I20260924 13:50:02.183917 26331 webserver.cc:460] Webserver started at http://127.13.149.193:44323/ using document root <none> and password file <none>
I20260924 13:50:02.184185 26331 fs_manager.cc:359] Metadata directory not provided
I20260924 13:50:02.184237 26331 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 13:50:02.184402 26331 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 24 13:50:02 dist-test-slave-sjl2 krb5kdc[25774](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257802, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.149.193@KRBTEST.COM for HTTP/127.13.149.212@KRBTEST.COM
I20260924 13:50:02.204171 26331 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790257629041645-13911-0/minicluster-data/ts-0/data/instance:
uuid: "b38a33372efe47a28d157e2781f2c069"
format_stamp: "Formatted at 2026-09-24 13:50:02 on dist-test-slave-sjl2"
server_key: "32f209b01ed24f86b06efe728e78d057"
server_key_iv: "d89442036f5b698933d5b0f295553f05"
server_key_version: "kuduclusterkey@0"
I20260924 13:50:02.204653 26331 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790257629041645-13911-0/minicluster-data/ts-0/wal/instance:
uuid: "b38a33372efe47a28d157e2781f2c069"
format_stamp: "Formatted at 2026-09-24 13:50:02 on dist-test-slave-sjl2"
server_key: "32f209b01ed24f86b06efe728e78d057"
server_key_iv: "d89442036f5b698933d5b0f295553f05"
server_key_version: "kuduclusterkey@0"
I20260924 13:50:02.206485 26331 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.001s
I20260924 13:50:02.222016 26346 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 13:50:02.222199 26331 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260924 13:50:02.222287 26331 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790257629041645-13911-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790257629041645-13911-0/minicluster-data/ts-0/wal
uuid: "b38a33372efe47a28d157e2781f2c069"
format_stamp: "Formatted at 2026-09-24 13:50:02 on dist-test-slave-sjl2"
server_key: "32f209b01ed24f86b06efe728e78d057"
server_key_iv: "d89442036f5b698933d5b0f295553f05"
server_key_version: "kuduclusterkey@0"
I20260924 13:50:02.222393 26331 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790257629041645-13911-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790257629041645-13911-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790257629041645-13911-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 13:50:02.253333 26331 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 13:50:02.256069 26331 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 13:50:02.256328 26331 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 13:50:02.256654 26331 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 13:50:02.257125 26331 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 13:50:02.257215 26331 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 13:50:02.257280 26331 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 13:50:02.257325 26331 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 13:50:02.265461 26331 rpc_server.cc:307] RPC server started. Bound to: 127.13.149.193:38631
I20260924 13:50:02.265700 26461 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.149.193:38631 every 8 connection(s)
I20260924 13:50:02.265996 26331 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790257629041645-13911-0/minicluster-data/ts-0/data/info.pb
I20260924 13:50:02.273955 13911 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJN9WRF/build/release/bin/kudu as pid 26331
I20260924 13:50:02.274055 13911 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790257629041645-13911-0/minicluster-data/ts-0/wal/instance
I20260924 13:50:02.274221 13911 external_mini_cluster.cc:1428] Setting key 18d8239a34f865ac9a44d458a452fa7d
Sep 24 13:50:02 dist-test-slave-sjl2 krb5kdc[25774](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257802, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.149.193@KRBTEST.COM for kudu/127.13.149.254@KRBTEST.COM
I20260924 13:50:02.279649 26464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:50:02.266810 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:34283 (local address 127.13.149.193:60789)
0924 13:50:02.267728 (+ 918us) negotiation.cc:107] Waiting for socket to connect
0924 13:50:02.267787 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0924 13:50:02.268432 (+ 645us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:50:02.268887 (+ 455us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:50:02.268892 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:50:02.269205 (+ 313us) client_negotiation.cc:189] Negotiated authn=SASL
0924 13:50:02.269513 (+ 308us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:50:02.269521 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:50:02.270550 (+ 1029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:50:02.270551 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:50:02.271857 (+ 1306us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:50:02.271861 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:50:02.272433 (+ 572us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:50:02.272434 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:50:02.272581 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:50:02.273235 (+ 654us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 13:50:02.273246 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0924 13:50:02.275906 (+ 2660us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 13:50:02.277940 (+ 2034us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 13:50:02.277952 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 13:50:02.277955 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0924 13:50:02.278301 (+ 346us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 13:50:02.278519 (+ 218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 13:50:02.278519 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 13:50:02.278519 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 13:50:02.278653 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 13:50:02.278937 (+ 284us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 13:50:02.278943 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 13:50:02.279122 (+ 179us) client_negotiation.cc:769] Sending connection context
0924 13:50:02.279241 (+ 119us) client_negotiation.cc:240] Negotiation successful
0924 13:50:02.279407 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":864,"thread_start_us":76,"threads_started":1}
I20260924 13:50:02.280229 26465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:50:02.266839 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.149.193:60789 (local address 127.13.149.254:34283)
0924 13:50:02.268633 (+ 1794us) server_negotiation.cc:206] Beginning negotiation
0924 13:50:02.268635 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:50:02.268643 (+ 8us) server_negotiation.cc:407] Connection header received
0924 13:50:02.268671 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:50:02.268672 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:50:02.268683 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:50:02.268745 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:50:02.269639 (+ 894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:50:02.270433 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:50:02.271971 (+ 1538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:50:02.272279 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:50:02.272347 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:50:02.276052 (+ 3705us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:50:02.276054 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:50:02.276054 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 13:50:02.276064 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:50:02.277823 (+ 1759us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:50:02.278388 (+ 565us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:50:02.278389 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:50:02.278390 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:50:02.278458 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:50:02.278716 (+ 258us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:50:02.278716 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:50:02.278717 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:50:02.278859 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:50:02.278914 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0924 13:50:02.280031 (+ 1117us) server_negotiation.cc:299] Negotiation successful
0924 13:50:02.280124 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1759,"thread_start_us":74,"threads_started":1}
I20260924 13:50:02.283097 26462 heartbeater.cc:347] Connected to a master server at 127.13.149.254:34283
I20260924 13:50:02.283284 26462 heartbeater.cc:464] Registering TS with master...
I20260924 13:50:02.283532 26462 heartbeater.cc:511] Master 127.13.149.254:34283 requested a full tablet report, sending...
I20260924 13:50:02.284112 26250 ts_manager.cc:194] Registered new tserver with Master: b38a33372efe47a28d157e2781f2c069 (127.13.149.193:38631)
I20260924 13:50:02.285023 26250 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.149.193@KRBTEST.COM'} at 127.13.149.193:60789
WARNING: no policy specified for kudu/127.13.149.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.149.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.149.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.194.keytab.
Entry for principal kudu/127.13.149.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.194.keytab.
I20260924 13:50:02.321004 13911 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJN9WRF/build/release/bin/kudu
/tmp/dist-test-taskJN9WRF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790257629041645-13911-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790257629041645-13911-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790257629041645-13911-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790257629041645-13911-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.13.149.212:55543/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.149.194:0
--local_ip_for_outbound_sockets=127.13.149.194
--webserver_interface=127.13.149.194
--webserver_port=0
--tserver_master_addrs=127.13.149.254:34283
--keytab_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.194.keytab
--principal=kudu/127.13.149.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.149.212:38701
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790257629041645-13911-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790257629041645-13911-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790257629041645-13911-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 13:50:02.452749 26468 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 13:50:02.453151 26468 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 13:50:02.453250 26468 flags.cc:425] Enabled unsafe flag: --never_fsync=true
Sep 24 13:50:02 dist-test-slave-sjl2 krb5kdc[25774](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790257802, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.149.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260924 13:50:02.455013 26468 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 13:50:02.455129 26468 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 13:50:02.455216 26468 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.149.194
I20260924 13:50:02.457635 26468 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.149.212:38701
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790257629041645-13911-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790257629041645-13911-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.13.149.212:55543/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.194.keytab
--principal=kudu/127.13.149.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.149.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790257629041645-13911-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.149.194
--webserver_port=0
--tserver_master_addrs=127.13.149.254:34283
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26468
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.149.194
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790257629041645-13911-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 12:29:37 UTC on ccycloud.quasar-iqzbxn.root.comops.site
build id 4670106
I20260924 13:50:02.458130 26468 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 13:50:02.458458 26468 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 13:50:02.464991 26475 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 13:50:02.464991 26476 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 13:50:02.465184 26478 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 13:50:02.466553 26468 server_base.cc:1034] running on GCE node
I20260924 13:50:02.466789 26468 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 13:50:02.467074 26468 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 13:50:02.483278 26468 hybrid_clock.cc:648] HybridClock initialized: now 1790257802483239 us; error 87 us; skew 500 ppm
Sep 24 13:50:02 dist-test-slave-sjl2 krb5kdc[25774](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257802, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.149.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 13:50:02.486740 26468 init.cc:377] Logged in from keytab as kudu/127.13.149.194@KRBTEST.COM (short username kudu)
I20260924 13:50:02.487320 26468 webserver.cc:460] Webserver started at http://127.13.149.194:34341/ using document root <none> and password file <none>
I20260924 13:50:02.487579 26468 fs_manager.cc:359] Metadata directory not provided
I20260924 13:50:02.487627 26468 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 13:50:02.487747 26468 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 24 13:50:02 dist-test-slave-sjl2 krb5kdc[25774](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257802, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.149.194@KRBTEST.COM for HTTP/127.13.149.212@KRBTEST.COM
I20260924 13:50:02.503805 26468 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790257629041645-13911-0/minicluster-data/ts-1/data/instance:
uuid: "ba730d9d9a2d4f8b87161cec9bcee786"
format_stamp: "Formatted at 2026-09-24 13:50:02 on dist-test-slave-sjl2"
server_key: "bdf38624b7c9912790feb4e0ad11d75e"
server_key_iv: "58bc4bf8bbc8fe9ed0180b8335dac282"
server_key_version: "kuduclusterkey@0"
I20260924 13:50:02.504359 26468 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790257629041645-13911-0/minicluster-data/ts-1/wal/instance:
uuid: "ba730d9d9a2d4f8b87161cec9bcee786"
format_stamp: "Formatted at 2026-09-24 13:50:02 on dist-test-slave-sjl2"
server_key: "bdf38624b7c9912790feb4e0ad11d75e"
server_key_iv: "58bc4bf8bbc8fe9ed0180b8335dac282"
server_key_version: "kuduclusterkey@0"
I20260924 13:50:02.506211 26468 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.001s sys 0.000s
I20260924 13:50:02.524257 26485 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 13:50:02.524492 26468 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260924 13:50:02.524581 26468 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790257629041645-13911-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790257629041645-13911-0/minicluster-data/ts-1/wal
uuid: "ba730d9d9a2d4f8b87161cec9bcee786"
format_stamp: "Formatted at 2026-09-24 13:50:02 on dist-test-slave-sjl2"
server_key: "bdf38624b7c9912790feb4e0ad11d75e"
server_key_iv: "58bc4bf8bbc8fe9ed0180b8335dac282"
server_key_version: "kuduclusterkey@0"
I20260924 13:50:02.524674 26468 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790257629041645-13911-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790257629041645-13911-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790257629041645-13911-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 13:50:02.541554 26468 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 13:50:02.543891 26468 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 13:50:02.544107 26468 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 13:50:02.544386 26468 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 13:50:02.544812 26468 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 13:50:02.544911 26468 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 13:50:02.545012 26468 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 13:50:02.545064 26468 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 13:50:02.551388 26468 rpc_server.cc:307] RPC server started. Bound to: 127.13.149.194:33273
I20260924 13:50:02.551880 26468 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790257629041645-13911-0/minicluster-data/ts-1/data/info.pb
I20260924 13:50:02.552476 13911 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJN9WRF/build/release/bin/kudu as pid 26468
I20260924 13:50:02.552579 13911 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790257629041645-13911-0/minicluster-data/ts-1/wal/instance
I20260924 13:50:02.552779 13911 external_mini_cluster.cc:1428] Setting key 97d9ac0e9de3bb0dbad49eca873bfd74
I20260924 13:50:02.554131 26599 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.149.194:33273 every 8 connection(s)
Sep 24 13:50:02 dist-test-slave-sjl2 krb5kdc[25774](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257802, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.149.194@KRBTEST.COM for kudu/127.13.149.254@KRBTEST.COM
I20260924 13:50:02.574074 26465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:50:02.555532 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.149.194:47505 (local address 127.13.149.254:34283)
0924 13:50:02.555593 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0924 13:50:02.555595 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:50:02.557266 (+ 1671us) server_negotiation.cc:407] Connection header received
0924 13:50:02.561831 (+ 4565us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:50:02.561833 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:50:02.561853 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:50:02.561884 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:50:02.562807 (+ 923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:50:02.563633 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:50:02.565220 (+ 1587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:50:02.565527 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:50:02.565570 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:50:02.570735 (+ 5165us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:50:02.570737 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:50:02.570738 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 13:50:02.570746 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:50:02.572332 (+ 1586us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:50:02.572860 (+ 528us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:50:02.572862 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:50:02.572862 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:50:02.572964 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:50:02.573272 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:50:02.573273 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:50:02.573273 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:50:02.573433 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:50:02.573485 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0924 13:50:02.573873 (+ 388us) server_negotiation.cc:299] Negotiation successful
0924 13:50:02.573975 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260924 13:50:02.574616 26602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:50:02.555263 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:34283 (local address 127.13.149.194:47505)
0924 13:50:02.557131 (+ 1868us) negotiation.cc:107] Waiting for socket to connect
0924 13:50:02.557182 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0924 13:50:02.561668 (+ 4486us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:50:02.562107 (+ 439us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:50:02.562113 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:50:02.562385 (+ 272us) client_negotiation.cc:189] Negotiated authn=SASL
0924 13:50:02.562685 (+ 300us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:50:02.562692 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:50:02.563760 (+ 1068us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:50:02.563761 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:50:02.565112 (+ 1351us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:50:02.565115 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:50:02.565680 (+ 565us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:50:02.565681 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:50:02.565827 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:50:02.568746 (+ 2919us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 13:50:02.568758 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0924 13:50:02.570604 (+ 1846us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 13:50:02.572429 (+ 1825us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 13:50:02.572438 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 13:50:02.572441 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0924 13:50:02.572748 (+ 307us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 13:50:02.573049 (+ 301us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 13:50:02.573050 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 13:50:02.573050 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 13:50:02.573199 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 13:50:02.573520 (+ 321us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 13:50:02.573526 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 13:50:02.573706 (+ 180us) client_negotiation.cc:769] Sending connection context
0924 13:50:02.574180 (+ 474us) client_negotiation.cc:240] Negotiation successful
0924 13:50:02.574375 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1818,"thread_start_us":86,"threads_started":1}
I20260924 13:50:02.575284 26600 heartbeater.cc:347] Connected to a master server at 127.13.149.254:34283
I20260924 13:50:02.575382 26600 heartbeater.cc:464] Registering TS with master...
I20260924 13:50:02.575567 26600 heartbeater.cc:511] Master 127.13.149.254:34283 requested a full tablet report, sending...
I20260924 13:50:02.576014 26250 ts_manager.cc:194] Registered new tserver with Master: ba730d9d9a2d4f8b87161cec9bcee786 (127.13.149.194:33273)
I20260924 13:50:02.576515 26250 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.149.194@KRBTEST.COM'} at 127.13.149.194:47505
WARNING: no policy specified for kudu/127.13.149.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.149.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.149.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.195.keytab.
Entry for principal kudu/127.13.149.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.195.keytab.
I20260924 13:50:02.597026 13911 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJN9WRF/build/release/bin/kudu
/tmp/dist-test-taskJN9WRF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790257629041645-13911-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790257629041645-13911-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790257629041645-13911-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790257629041645-13911-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.13.149.212:55543/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.149.195:0
--local_ip_for_outbound_sockets=127.13.149.195
--webserver_interface=127.13.149.195
--webserver_port=0
--tserver_master_addrs=127.13.149.254:34283
--keytab_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.195.keytab
--principal=kudu/127.13.149.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.149.212:38701
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790257629041645-13911-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790257629041645-13911-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790257629041645-13911-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 13:50:02.726557 26605 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 13:50:02.726912 26605 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 13:50:02.726996 26605 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 13:50:02.729220 26605 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 13:50:02.729390 26605 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 13:50:02.729488 26605 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.149.195
I20260924 13:50:02.732241 26605 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.149.212:38701
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790257629041645-13911-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790257629041645-13911-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.13.149.212:55543/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.195.keytab
--principal=kudu/127.13.149.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.149.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790257629041645-13911-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.149.195
--webserver_port=0
--tserver_master_addrs=127.13.149.254:34283
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26605
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.149.195
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790257629041645-13911-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 12:29:37 UTC on ccycloud.quasar-iqzbxn.root.comops.site
build id 4670106
I20260924 13:50:02.732801 26605 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 13:50:02.733174 26605 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 13:50:02.739909 26611 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 13:50:02.740255 26610 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 13:50:02.740613 26605 server_base.cc:1034] running on GCE node
W20260924 13:50:02.740765 26613 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 13:50:02.741048 26605 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 13:50:02.741317 26605 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 13:50:02.742480 26605 hybrid_clock.cc:648] HybridClock initialized: now 1790257802742460 us; error 32 us; skew 500 ppm
Sep 24 13:50:02 dist-test-slave-sjl2 krb5kdc[25774](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257802, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.149.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 13:50:02.745797 26605 init.cc:377] Logged in from keytab as kudu/127.13.149.195@KRBTEST.COM (short username kudu)
I20260924 13:50:02.746347 26605 webserver.cc:460] Webserver started at http://127.13.149.195:46369/ using document root <none> and password file <none>
I20260924 13:50:02.746596 26605 fs_manager.cc:359] Metadata directory not provided
I20260924 13:50:02.746640 26605 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 13:50:02.746758 26605 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 24 13:50:02 dist-test-slave-sjl2 krb5kdc[25774](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257802, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.149.195@KRBTEST.COM for HTTP/127.13.149.212@KRBTEST.COM
I20260924 13:50:02.760097 26605 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790257629041645-13911-0/minicluster-data/ts-2/data/instance:
uuid: "a9eb7fbc08de4cc78709a9e6f9b7823d"
format_stamp: "Formatted at 2026-09-24 13:50:02 on dist-test-slave-sjl2"
server_key: "b861d5b1c453f669928b0f18954e9eca"
server_key_iv: "e3ee9b2af6103e2733bd42413e94ee82"
server_key_version: "kuduclusterkey@0"
I20260924 13:50:02.760519 26605 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790257629041645-13911-0/minicluster-data/ts-2/wal/instance:
uuid: "a9eb7fbc08de4cc78709a9e6f9b7823d"
format_stamp: "Formatted at 2026-09-24 13:50:02 on dist-test-slave-sjl2"
server_key: "b861d5b1c453f669928b0f18954e9eca"
server_key_iv: "e3ee9b2af6103e2733bd42413e94ee82"
server_key_version: "kuduclusterkey@0"
I20260924 13:50:02.762348 26605 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.004s
I20260924 13:50:02.779739 26620 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 13:50:02.780069 26605 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.000s
I20260924 13:50:02.780220 26605 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790257629041645-13911-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790257629041645-13911-0/minicluster-data/ts-2/wal
uuid: "a9eb7fbc08de4cc78709a9e6f9b7823d"
format_stamp: "Formatted at 2026-09-24 13:50:02 on dist-test-slave-sjl2"
server_key: "b861d5b1c453f669928b0f18954e9eca"
server_key_iv: "e3ee9b2af6103e2733bd42413e94ee82"
server_key_version: "kuduclusterkey@0"
I20260924 13:50:02.780392 26605 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790257629041645-13911-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790257629041645-13911-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790257629041645-13911-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 13:50:02.801425 26605 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 13:50:02.803790 26605 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 13:50:02.803977 26605 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 13:50:02.804253 26605 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 13:50:02.804631 26605 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 13:50:02.804724 26605 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 13:50:02.804786 26605 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 13:50:02.804833 26605 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 13:50:02.810782 26605 rpc_server.cc:307] RPC server started. Bound to: 127.13.149.195:46435
I20260924 13:50:02.811252 26605 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790257629041645-13911-0/minicluster-data/ts-2/data/info.pb
I20260924 13:50:02.817329 13911 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJN9WRF/build/release/bin/kudu as pid 26605
I20260924 13:50:02.817444 13911 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790257629041645-13911-0/minicluster-data/ts-2/wal/instance
I20260924 13:50:02.817632 13911 external_mini_cluster.cc:1428] Setting key 924bff9bee79dc43b8a12532bf64b4e0
I20260924 13:50:02.826243 26733 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.149.195:46435 every 8 connection(s)
Sep 24 13:50:02 dist-test-slave-sjl2 krb5kdc[25774](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257802, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.149.195@KRBTEST.COM for kudu/127.13.149.254@KRBTEST.COM
I20260924 13:50:02.844504 26465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:50:02.825945 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.149.195:59461 (local address 127.13.149.254:34283)
0924 13:50:02.826003 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0924 13:50:02.826005 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:50:02.827839 (+ 1834us) server_negotiation.cc:407] Connection header received
0924 13:50:02.829291 (+ 1452us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:50:02.829292 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:50:02.829304 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:50:02.829329 (+ 25us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:50:02.830596 (+ 1267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:50:02.831335 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:50:02.836440 (+ 5105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:50:02.836750 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:50:02.836795 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:50:02.841045 (+ 4250us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:50:02.841047 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:50:02.841047 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 13:50:02.841055 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:50:02.842613 (+ 1558us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:50:02.843123 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:50:02.843124 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:50:02.843124 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:50:02.843183 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:50:02.843773 (+ 590us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:50:02.843774 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:50:02.843775 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:50:02.843918 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:50:02.843967 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0924 13:50:02.844333 (+ 366us) server_negotiation.cc:299] Negotiation successful
0924 13:50:02.844415 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260924 13:50:02.845295 26736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:50:02.825955 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:34283 (local address 127.13.149.195:59461)
0924 13:50:02.827670 (+ 1715us) negotiation.cc:107] Waiting for socket to connect
0924 13:50:02.827753 (+ 83us) client_negotiation.cc:174] Beginning negotiation
0924 13:50:02.828579 (+ 826us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:50:02.829514 (+ 935us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:50:02.829518 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:50:02.829773 (+ 255us) client_negotiation.cc:189] Negotiated authn=SASL
0924 13:50:02.830491 (+ 718us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:50:02.830499 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:50:02.831442 (+ 943us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:50:02.831443 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:50:02.836336 (+ 4893us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:50:02.836340 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:50:02.836903 (+ 563us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:50:02.836904 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:50:02.837078 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:50:02.837699 (+ 621us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 13:50:02.837709 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0924 13:50:02.840942 (+ 3233us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 13:50:02.842711 (+ 1769us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 13:50:02.842717 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 13:50:02.842720 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0924 13:50:02.843041 (+ 321us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 13:50:02.843245 (+ 204us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 13:50:02.843246 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 13:50:02.843246 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 13:50:02.843690 (+ 444us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 13:50:02.844001 (+ 311us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 13:50:02.844007 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 13:50:02.844175 (+ 168us) client_negotiation.cc:769] Sending connection context
0924 13:50:02.844846 (+ 671us) client_negotiation.cc:240] Negotiation successful
0924 13:50:02.845053 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1657,"thread_start_us":109,"threads_started":1}
I20260924 13:50:02.845922 26734 heartbeater.cc:347] Connected to a master server at 127.13.149.254:34283
I20260924 13:50:02.846014 26734 heartbeater.cc:464] Registering TS with master...
I20260924 13:50:02.846195 26734 heartbeater.cc:511] Master 127.13.149.254:34283 requested a full tablet report, sending...
I20260924 13:50:02.846585 26250 ts_manager.cc:194] Registered new tserver with Master: a9eb7fbc08de4cc78709a9e6f9b7823d (127.13.149.195:46435)
I20260924 13:50:02.847047 26250 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.149.195@KRBTEST.COM'} at 127.13.149.195:59461
I20260924 13:50:02.850433 13911 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260924 13:50:02.851112 13911 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJN9WRF/build/release/bin/kudu with pid 26331
I20260924 13:50:02.857945 13911 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJN9WRF/build/release/bin/kudu with pid 26468
I20260924 13:50:02.864940 13911 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJN9WRF/build/release/bin/kudu with pid 26605
I20260924 13:50:02.870196 13911 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJN9WRF/build/release/bin/kudu with pid 26218
I20260924 13:50:02.875479 13911 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJN9WRF/build/release/bin/kudu
/tmp/dist-test-taskJN9WRF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790257629041645-13911-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790257629041645-13911-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790257629041645-13911-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790257629041645-13911-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.149.254:34283
--webserver_interface=127.13.149.254
--webserver_port=42081
--keytab_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.254.keytab
--principal=kudu/127.13.149.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.149.212:38701
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.149.254:34283
--ranger_config_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790257629041645-13911-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.13.149.212:55543/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.13.149.212:55543/kms with env {KRB5CCNAME=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790257629041645-13911-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790257629041645-13911-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790257629041645-13911-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 13:50:02.955681 26737 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 13:50:02.955888 26737 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 13:50:02.955909 26737 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 13:50:02.957079 26737 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 13:50:02.957123 26737 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 13:50:02.957136 26737 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 13:50:02.957155 26737 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 13:50:02.957167 26737 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 13:50:02.958739 26737 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.149.212:38701
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790257629041645-13911-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790257629041645-13911-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.13.149.212:55543/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.13.149.254:34283
--ranger_config_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790257629041645-13911-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.254.keytab
--principal=kudu/127.13.149.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.149.254:34283
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790257629041645-13911-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.149.254
--webserver_port=42081
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26737
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790257629041645-13911-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 12:29:37 UTC on ccycloud.quasar-iqzbxn.root.comops.site
build id 4670106
I20260924 13:50:02.958959 26737 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 13:50:02.959169 26737 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260924 13:50:02.963965 26737 server_base.cc:1034] running on GCE node
W20260924 13:50:02.963933 26743 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 13:50:02.963924 26742 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 13:50:02.963934 26745 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 13:50:02.964306 26737 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 13:50:02.964572 26737 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 13:50:02.965711 26737 hybrid_clock.cc:648] HybridClock initialized: now 1790257802965699 us; error 28 us; skew 500 ppm
Sep 24 13:50:02 dist-test-slave-sjl2 krb5kdc[25774](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257802, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.149.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 13:50:02.967958 26737 init.cc:377] Logged in from keytab as kudu/127.13.149.254@KRBTEST.COM (short username kudu)
I20260924 13:50:02.968365 26737 webserver.cc:460] Webserver started at http://127.13.149.254:42081/ using document root <none> and password file <none>
I20260924 13:50:02.968562 26737 fs_manager.cc:359] Metadata directory not provided
I20260924 13:50:02.968616 26737 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 24 13:50:02 dist-test-slave-sjl2 krb5kdc[25774](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257802, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.149.254@KRBTEST.COM for HTTP/127.13.149.212@KRBTEST.COM
I20260924 13:50:03.013365 26737 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260924 13:50:03.016794 26753 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 13:50:03.017143 26737 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260924 13:50:03.017292 26737 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790257629041645-13911-0/minicluster-data/master-0/data,/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790257629041645-13911-0/minicluster-data/master-0/wal
uuid: "9bb72cc199554d53af2e97d01120dbd8"
format_stamp: "Formatted at 2026-09-24 13:50:01 on dist-test-slave-sjl2"
server_key: "e075b27eb8dc25a7f89cad91fd00701e"
server_key_iv: "e220485a69ecf78a84757a523475fce2"
server_key_version: "kuduclusterkey@0"
I20260924 13:50:03.017714 26737 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790257629041645-13911-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790257629041645-13911-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790257629041645-13911-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 13:50:03.032621 26737 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 13:50:03.034878 26737 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 13:50:03.035041 26737 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 13:50:03.039001 26737 rpc_server.cc:307] RPC server started. Bound to: 127.13.149.254:34283
I20260924 13:50:03.039091 26805 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.149.254:34283 every 8 connection(s)
I20260924 13:50:03.039429 26737 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790257629041645-13911-0/minicluster-data/master-0/data/info.pb
I20260924 13:50:03.040630 26806 sys_catalog.cc:263] Verifying existing consensus state
I20260924 13:50:03.041319 26806 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9bb72cc199554d53af2e97d01120dbd8: Bootstrap starting.
I20260924 13:50:03.041615 13911 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJN9WRF/build/release/bin/kudu as pid 26737
I20260924 13:50:03.041882 13911 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJN9WRF/build/release/bin/kudu
/tmp/dist-test-taskJN9WRF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790257629041645-13911-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790257629041645-13911-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790257629041645-13911-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790257629041645-13911-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.13.149.212:55543/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.149.193:38631
--local_ip_for_outbound_sockets=127.13.149.193
--tserver_master_addrs=127.13.149.254:34283
--webserver_port=44323
--webserver_interface=127.13.149.193
--keytab_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.193.keytab
--principal=kudu/127.13.149.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.149.212:38701
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.13.149.212:55543/kms with env {KRB5CCNAME=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790257629041645-13911-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790257629041645-13911-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790257629041645-13911-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260924 13:50:03.049499 26806 log.cc:824] T 00000000000000000000000000000000 P 9bb72cc199554d53af2e97d01120dbd8: Log is configured to *not* fsync() on all Append() calls
I20260924 13:50:03.051112 26806 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9bb72cc199554d53af2e97d01120dbd8: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260924 13:50:03.051359 26806 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9bb72cc199554d53af2e97d01120dbd8: Bootstrap complete.
I20260924 13:50:03.053575 26806 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9bb72cc199554d53af2e97d01120dbd8 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9bb72cc199554d53af2e97d01120dbd8" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 34283 } }
I20260924 13:50:03.053812 26806 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9bb72cc199554d53af2e97d01120dbd8 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9bb72cc199554d53af2e97d01120dbd8, State: Initialized, Role: FOLLOWER
I20260924 13:50:03.053972 26806 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9bb72cc199554d53af2e97d01120dbd8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9bb72cc199554d53af2e97d01120dbd8" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 34283 } }
I20260924 13:50:03.054028 26806 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9bb72cc199554d53af2e97d01120dbd8 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 13:50:03.054073 26806 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9bb72cc199554d53af2e97d01120dbd8 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 13:50:03.054116 26806 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9bb72cc199554d53af2e97d01120dbd8 [term 1 FOLLOWER]: Advancing to term 2
I20260924 13:50:03.055065 26806 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9bb72cc199554d53af2e97d01120dbd8 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9bb72cc199554d53af2e97d01120dbd8" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 34283 } }
I20260924 13:50:03.055167 26806 leader_election.cc:302] T 00000000000000000000000000000000 P 9bb72cc199554d53af2e97d01120dbd8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9bb72cc199554d53af2e97d01120dbd8; no voters:
I20260924 13:50:03.055330 26806 leader_election.cc:288] T 00000000000000000000000000000000 P 9bb72cc199554d53af2e97d01120dbd8 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260924 13:50:03.055365 26810 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9bb72cc199554d53af2e97d01120dbd8 [term 2 FOLLOWER]: Leader election won for term 2
I20260924 13:50:03.055464 26810 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9bb72cc199554d53af2e97d01120dbd8 [term 2 LEADER]: Becoming Leader. State: Replica: 9bb72cc199554d53af2e97d01120dbd8, State: Running, Role: LEADER
I20260924 13:50:03.055585 26806 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9bb72cc199554d53af2e97d01120dbd8 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 13:50:03.055619 26810 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9bb72cc199554d53af2e97d01120dbd8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9bb72cc199554d53af2e97d01120dbd8" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 34283 } }
I20260924 13:50:03.055987 26812 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9bb72cc199554d53af2e97d01120dbd8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9bb72cc199554d53af2e97d01120dbd8. Latest consensus state: current_term: 2 leader_uuid: "9bb72cc199554d53af2e97d01120dbd8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9bb72cc199554d53af2e97d01120dbd8" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 34283 } } }
I20260924 13:50:03.056058 26812 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9bb72cc199554d53af2e97d01120dbd8 [sys.catalog]: This master's current role is: LEADER
I20260924 13:50:03.056246 26811 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9bb72cc199554d53af2e97d01120dbd8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "9bb72cc199554d53af2e97d01120dbd8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9bb72cc199554d53af2e97d01120dbd8" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 34283 } } }
I20260924 13:50:03.056308 26811 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9bb72cc199554d53af2e97d01120dbd8 [sys.catalog]: This master's current role is: LEADER
I20260924 13:50:03.057065 26815 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 13:50:03.057068 26806 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790257629041645-13911-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260924 13:50:03.059031 26815 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 13:50:03.059268 26815 catalog_manager.cc:1259] Loaded cluster ID: ba7fab592cd54f67ab6466657dfe587d
I20260924 13:50:03.059307 26815 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 13:50:03.061161 26815 catalog_manager.cc:1504] Loading token signing keys...
I20260924 13:50:03.061363 26815 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 9bb72cc199554d53af2e97d01120dbd8: Loaded TSK: 0
I20260924 13:50:03.061583 26815 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260924 13:50:03.129743 26808 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 13:50:03.130125 26808 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 13:50:03.130185 26808 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 13:50:03.131763 26808 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 13:50:03.131812 26808 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 13:50:03.131841 26808 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.149.193
I20260924 13:50:03.133615 26808 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.149.212:38701
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790257629041645-13911-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790257629041645-13911-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.13.149.212:55543/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.193.keytab
--principal=kudu/127.13.149.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.149.193:38631
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790257629041645-13911-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.149.193
--webserver_port=44323
--tserver_master_addrs=127.13.149.254:34283
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26808
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.149.193
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790257629041645-13911-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 12:29:37 UTC on ccycloud.quasar-iqzbxn.root.comops.site
build id 4670106
I20260924 13:50:03.133900 26808 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 13:50:03.134229 26808 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 13:50:03.140512 26843 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 13:50:03.140486 26846 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 13:50:03.141018 26844 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 13:50:03.141491 26808 server_base.cc:1034] running on GCE node
I20260924 13:50:03.141716 26808 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 13:50:03.142006 26808 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 13:50:03.143178 26808 hybrid_clock.cc:648] HybridClock initialized: now 1790257803143161 us; error 31 us; skew 500 ppm
Sep 24 13:50:03 dist-test-slave-sjl2 krb5kdc[25774](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257803, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.149.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 13:50:03.146421 26808 init.cc:377] Logged in from keytab as kudu/127.13.149.193@KRBTEST.COM (short username kudu)
I20260924 13:50:03.146972 26808 webserver.cc:460] Webserver started at http://127.13.149.193:44323/ using document root <none> and password file <none>
I20260924 13:50:03.147212 26808 fs_manager.cc:359] Metadata directory not provided
I20260924 13:50:03.147279 26808 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 24 13:50:03 dist-test-slave-sjl2 krb5kdc[25774](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257803, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.149.193@KRBTEST.COM for HTTP/127.13.149.212@KRBTEST.COM
I20260924 13:50:03.170006 26808 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260924 13:50:03.170900 26854 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 13:50:03.171068 26808 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260924 13:50:03.171144 26808 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790257629041645-13911-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790257629041645-13911-0/minicluster-data/ts-0/wal
uuid: "b38a33372efe47a28d157e2781f2c069"
format_stamp: "Formatted at 2026-09-24 13:50:02 on dist-test-slave-sjl2"
server_key: "32f209b01ed24f86b06efe728e78d057"
server_key_iv: "d89442036f5b698933d5b0f295553f05"
server_key_version: "kuduclusterkey@0"
I20260924 13:50:03.171581 26808 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790257629041645-13911-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790257629041645-13911-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790257629041645-13911-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 13:50:03.184231 26808 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 13:50:03.186290 26808 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 13:50:03.186432 26808 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 13:50:03.186671 26808 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 13:50:03.187131 26808 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 13:50:03.187170 26808 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 13:50:03.187201 26808 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 13:50:03.187237 26808 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 13:50:03.193933 26808 rpc_server.cc:307] RPC server started. Bound to: 127.13.149.193:38631
I20260924 13:50:03.194484 26808 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790257629041645-13911-0/minicluster-data/ts-0/data/info.pb
I20260924 13:50:03.194582 26967 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.149.193:38631 every 8 connection(s)
I20260924 13:50:03.203065 13911 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJN9WRF/build/release/bin/kudu as pid 26808
I20260924 13:50:03.203361 13911 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJN9WRF/build/release/bin/kudu
/tmp/dist-test-taskJN9WRF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790257629041645-13911-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790257629041645-13911-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790257629041645-13911-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790257629041645-13911-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.13.149.212:55543/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.149.194:33273
--local_ip_for_outbound_sockets=127.13.149.194
--tserver_master_addrs=127.13.149.254:34283
--webserver_port=34341
--webserver_interface=127.13.149.194
--keytab_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.194.keytab
--principal=kudu/127.13.149.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.149.212:38701
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.13.149.212:55543/kms with env {KRB5CCNAME=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790257629041645-13911-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790257629041645-13911-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790257629041645-13911-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 24 13:50:03 dist-test-slave-sjl2 krb5kdc[25774](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257803, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.149.193@KRBTEST.COM for kudu/127.13.149.254@KRBTEST.COM
I20260924 13:50:03.209991 26970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:50:03.198543 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:34283 (local address 127.13.149.193:60573)
0924 13:50:03.198991 (+ 448us) negotiation.cc:107] Waiting for socket to connect
0924 13:50:03.199033 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0924 13:50:03.199520 (+ 487us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:50:03.199973 (+ 453us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:50:03.199977 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:50:03.200268 (+ 291us) client_negotiation.cc:189] Negotiated authn=SASL
0924 13:50:03.200460 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:50:03.200465 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:50:03.201939 (+ 1474us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:50:03.201941 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:50:03.202718 (+ 777us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:50:03.202720 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:50:03.203642 (+ 922us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:50:03.203643 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:50:03.203776 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:50:03.204341 (+ 565us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 13:50:03.204351 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0924 13:50:03.205897 (+ 1546us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 13:50:03.208017 (+ 2120us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 13:50:03.208025 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 13:50:03.208028 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0924 13:50:03.208360 (+ 332us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 13:50:03.208642 (+ 282us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 13:50:03.208642 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 13:50:03.208643 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 13:50:03.208772 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 13:50:03.209302 (+ 530us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 13:50:03.209310 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 13:50:03.209466 (+ 156us) client_negotiation.cc:769] Sending connection context
0924 13:50:03.209574 (+ 108us) client_negotiation.cc:240] Negotiation successful
0924 13:50:03.209740 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":404,"thread_start_us":267,"threads_started":1}
I20260924 13:50:03.211061 26971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:50:03.198820 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.149.193:60573 (local address 127.13.149.254:34283)
0924 13:50:03.199007 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0924 13:50:03.199016 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0924 13:50:03.199122 (+ 106us) server_negotiation.cc:407] Connection header received
0924 13:50:03.199762 (+ 640us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:50:03.199767 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:50:03.199810 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:50:03.199920 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:50:03.200578 (+ 658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:50:03.201841 (+ 1263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:50:03.202803 (+ 962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:50:03.203415 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:50:03.203518 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:50:03.206029 (+ 2511us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:50:03.206034 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:50:03.206034 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 13:50:03.206047 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:50:03.207917 (+ 1870us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:50:03.208457 (+ 540us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:50:03.208460 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:50:03.208461 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:50:03.208573 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:50:03.208838 (+ 265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:50:03.208839 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:50:03.208839 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:50:03.209187 (+ 348us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:50:03.209273 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0924 13:50:03.210689 (+ 1416us) server_negotiation.cc:299] Negotiation successful
0924 13:50:03.210817 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":143,"thread_start_us":72,"threads_started":1}
I20260924 13:50:03.213503 26968 heartbeater.cc:347] Connected to a master server at 127.13.149.254:34283
I20260924 13:50:03.213604 26968 heartbeater.cc:464] Registering TS with master...
I20260924 13:50:03.213806 26968 heartbeater.cc:511] Master 127.13.149.254:34283 requested a full tablet report, sending...
W20260924 13:50:03.214764 26968 heartbeater.cc:644] Failed to heartbeat to 127.13.149.254:34283 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260924 13:50:03.214834 26968 heartbeater.cc:464] Registering TS with master...
I20260924 13:50:03.214897 26968 heartbeater.cc:511] Master 127.13.149.254:34283 requested a full tablet report, sending...
I20260924 13:50:03.215368 26968 heartbeater.cc:464] Registering TS with master...
I20260924 13:50:03.215441 26968 heartbeater.cc:511] Master 127.13.149.254:34283 requested a full tablet report, sending...
W20260924 13:50:03.215884 26968 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260924 13:50:03.307401 26972 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 13:50:03.307761 26972 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 13:50:03.307845 26972 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 13:50:03.309674 26972 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 13:50:03.309782 26972 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 13:50:03.309867 26972 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.149.194
I20260924 13:50:03.312067 26972 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.149.212:38701
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790257629041645-13911-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790257629041645-13911-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.13.149.212:55543/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.194.keytab
--principal=kudu/127.13.149.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.149.194:33273
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790257629041645-13911-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.149.194
--webserver_port=34341
--tserver_master_addrs=127.13.149.254:34283
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26972
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.149.194
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790257629041645-13911-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 12:29:37 UTC on ccycloud.quasar-iqzbxn.root.comops.site
build id 4670106
I20260924 13:50:03.312790 26972 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 13:50:03.313155 26972 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 13:50:03.319597 26977 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 13:50:03.319892 26978 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 13:50:03.319599 26980 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 13:50:03.320536 26972 server_base.cc:1034] running on GCE node
I20260924 13:50:03.320791 26972 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 13:50:03.321097 26972 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 13:50:03.322270 26972 hybrid_clock.cc:648] HybridClock initialized: now 1790257803322243 us; error 43 us; skew 500 ppm
Sep 24 13:50:03 dist-test-slave-sjl2 krb5kdc[25774](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257803, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.149.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 13:50:03.325014 26972 init.cc:377] Logged in from keytab as kudu/127.13.149.194@KRBTEST.COM (short username kudu)
I20260924 13:50:03.325608 26972 webserver.cc:460] Webserver started at http://127.13.149.194:34341/ using document root <none> and password file <none>
I20260924 13:50:03.325887 26972 fs_manager.cc:359] Metadata directory not provided
I20260924 13:50:03.325966 26972 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 24 13:50:03 dist-test-slave-sjl2 krb5kdc[25774](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257803, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.149.194@KRBTEST.COM for HTTP/127.13.149.212@KRBTEST.COM
I20260924 13:50:03.349402 26972 fs_manager.cc:709] Time spent opening directory manager: real 0.009s user 0.001s sys 0.000s
I20260924 13:50:03.350389 26988 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 13:50:03.350574 26972 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260924 13:50:03.350668 26972 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790257629041645-13911-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790257629041645-13911-0/minicluster-data/ts-1/wal
uuid: "ba730d9d9a2d4f8b87161cec9bcee786"
format_stamp: "Formatted at 2026-09-24 13:50:02 on dist-test-slave-sjl2"
server_key: "bdf38624b7c9912790feb4e0ad11d75e"
server_key_iv: "58bc4bf8bbc8fe9ed0180b8335dac282"
server_key_version: "kuduclusterkey@0"
I20260924 13:50:03.351081 26972 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790257629041645-13911-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790257629041645-13911-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790257629041645-13911-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 13:50:03.364032 26972 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 13:50:03.366529 26972 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 13:50:03.366693 26972 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 13:50:03.366937 26972 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 13:50:03.367326 26972 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 13:50:03.367374 26972 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 13:50:03.367405 26972 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 13:50:03.367432 26972 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 13:50:03.373643 26972 rpc_server.cc:307] RPC server started. Bound to: 127.13.149.194:33273
I20260924 13:50:03.373797 27101 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.149.194:33273 every 8 connection(s)
I20260924 13:50:03.374146 26972 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790257629041645-13911-0/minicluster-data/ts-1/data/info.pb
Sep 24 13:50:03 dist-test-slave-sjl2 krb5kdc[25774](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257803, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.149.194@KRBTEST.COM for kudu/127.13.149.254@KRBTEST.COM
I20260924 13:50:03.382164 13911 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJN9WRF/build/release/bin/kudu as pid 26972
I20260924 13:50:03.382444 13911 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJN9WRF/build/release/bin/kudu
/tmp/dist-test-taskJN9WRF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790257629041645-13911-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790257629041645-13911-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790257629041645-13911-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790257629041645-13911-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.13.149.212:55543/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.149.195:46435
--local_ip_for_outbound_sockets=127.13.149.195
--tserver_master_addrs=127.13.149.254:34283
--webserver_port=46369
--webserver_interface=127.13.149.195
--keytab_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.195.keytab
--principal=kudu/127.13.149.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.149.212:38701
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.13.149.212:55543/kms with env {KRB5CCNAME=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790257629041645-13911-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790257629041645-13911-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790257629041645-13911-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260924 13:50:03.385531 26971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:50:03.374892 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.149.194:36569 (local address 127.13.149.254:34283)
0924 13:50:03.374938 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0924 13:50:03.374940 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:50:03.375270 (+ 330us) server_negotiation.cc:407] Connection header received
0924 13:50:03.375813 (+ 543us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:50:03.375815 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:50:03.375830 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:50:03.375879 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:50:03.376419 (+ 540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:50:03.376996 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:50:03.378025 (+ 1029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:50:03.378228 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:50:03.378283 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:50:03.380449 (+ 2166us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:50:03.380450 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:50:03.380451 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 13:50:03.380460 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:50:03.381929 (+ 1469us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:50:03.384420 (+ 2491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:50:03.384421 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:50:03.384422 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:50:03.384485 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:50:03.384732 (+ 247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:50:03.384732 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:50:03.384733 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:50:03.384869 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:50:03.384976 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0924 13:50:03.385337 (+ 361us) server_negotiation.cc:299] Negotiation successful
0924 13:50:03.385435 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260924 13:50:03.385944 27104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:50:03.374913 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:34283 (local address 127.13.149.194:36569)
0924 13:50:03.375152 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0924 13:50:03.375185 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0924 13:50:03.375713 (+ 528us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:50:03.375947 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:50:03.375951 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:50:03.376137 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0924 13:50:03.376328 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:50:03.376333 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:50:03.377097 (+ 764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:50:03.377098 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:50:03.377937 (+ 839us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:50:03.377940 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:50:03.378322 (+ 382us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:50:03.378323 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:50:03.378445 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:50:03.378929 (+ 484us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 13:50:03.378937 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0924 13:50:03.380356 (+ 1419us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 13:50:03.384075 (+ 3719us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 13:50:03.384082 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 13:50:03.384084 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 13:50:03.384334 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 13:50:03.384541 (+ 207us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 13:50:03.384542 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 13:50:03.384542 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 13:50:03.384671 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 13:50:03.384966 (+ 295us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 13:50:03.384974 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 13:50:03.385175 (+ 201us) client_negotiation.cc:769] Sending connection context
0924 13:50:03.385314 (+ 139us) client_negotiation.cc:240] Negotiation successful
0924 13:50:03.385700 (+ 386us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":71,"threads_started":1}
I20260924 13:50:03.386492 27102 heartbeater.cc:347] Connected to a master server at 127.13.149.254:34283
I20260924 13:50:03.386597 27102 heartbeater.cc:464] Registering TS with master...
I20260924 13:50:03.386782 27102 heartbeater.cc:511] Master 127.13.149.254:34283 requested a full tablet report, sending...
W20260924 13:50:03.387424 27102 heartbeater.cc:644] Failed to heartbeat to 127.13.149.254:34283 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260924 13:50:03.387478 27102 heartbeater.cc:464] Registering TS with master...
I20260924 13:50:03.387527 27102 heartbeater.cc:511] Master 127.13.149.254:34283 requested a full tablet report, sending...
I20260924 13:50:03.387900 27102 heartbeater.cc:464] Registering TS with master...
I20260924 13:50:03.387979 27102 heartbeater.cc:511] Master 127.13.149.254:34283 requested a full tablet report, sending...
W20260924 13:50:03.388298 27102 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260924 13:50:03.485735 27105 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 13:50:03.486097 27105 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 13:50:03.486186 27105 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 13:50:03.487929 27105 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 13:50:03.488047 27105 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 13:50:03.488137 27105 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.149.195
I20260924 13:50:03.490419 27105 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.149.212:38701
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790257629041645-13911-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790257629041645-13911-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.13.149.212:55543/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.195.keytab
--principal=kudu/127.13.149.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.149.195:46435
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790257629041645-13911-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.149.195
--webserver_port=46369
--tserver_master_addrs=127.13.149.254:34283
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27105
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.149.195
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790257629041645-13911-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 12:29:37 UTC on ccycloud.quasar-iqzbxn.root.comops.site
build id 4670106
I20260924 13:50:03.490897 27105 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 13:50:03.491241 27105 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 13:50:03.497923 27113 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 13:50:03.497915 27110 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 13:50:03.498211 27111 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 13:50:03.498796 27105 server_base.cc:1034] running on GCE node
I20260924 13:50:03.498994 27105 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 13:50:03.499228 27105 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 13:50:03.500411 27105 hybrid_clock.cc:648] HybridClock initialized: now 1790257803500398 us; error 27 us; skew 500 ppm
Sep 24 13:50:03 dist-test-slave-sjl2 krb5kdc[25774](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257803, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.149.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 13:50:03.502923 27105 init.cc:377] Logged in from keytab as kudu/127.13.149.195@KRBTEST.COM (short username kudu)
I20260924 13:50:03.503405 27105 webserver.cc:460] Webserver started at http://127.13.149.195:46369/ using document root <none> and password file <none>
I20260924 13:50:03.503626 27105 fs_manager.cc:359] Metadata directory not provided
I20260924 13:50:03.503690 27105 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 24 13:50:03 dist-test-slave-sjl2 krb5kdc[25774](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257803, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.149.195@KRBTEST.COM for HTTP/127.13.149.212@KRBTEST.COM
I20260924 13:50:03.523602 27105 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260924 13:50:03.524464 27121 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 13:50:03.524603 27105 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260924 13:50:03.524693 27105 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790257629041645-13911-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790257629041645-13911-0/minicluster-data/ts-2/wal
uuid: "a9eb7fbc08de4cc78709a9e6f9b7823d"
format_stamp: "Formatted at 2026-09-24 13:50:02 on dist-test-slave-sjl2"
server_key: "b861d5b1c453f669928b0f18954e9eca"
server_key_iv: "e3ee9b2af6103e2733bd42413e94ee82"
server_key_version: "kuduclusterkey@0"
I20260924 13:50:03.525131 27105 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790257629041645-13911-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790257629041645-13911-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790257629041645-13911-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 13:50:03.553828 27105 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 13:50:03.556447 27105 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 13:50:03.556701 27105 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 13:50:03.557099 27105 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 13:50:03.557544 27105 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 13:50:03.557601 27105 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 13:50:03.557649 27105 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 13:50:03.557677 27105 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 13:50:03.564253 27105 rpc_server.cc:307] RPC server started. Bound to: 127.13.149.195:46435
I20260924 13:50:03.564757 27105 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790257629041645-13911-0/minicluster-data/ts-2/data/info.pb
I20260924 13:50:03.565871 27235 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.149.195:46435 every 8 connection(s)
Sep 24 13:50:03 dist-test-slave-sjl2 krb5kdc[25774](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257803, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.149.195@KRBTEST.COM for kudu/127.13.149.254@KRBTEST.COM
I20260924 13:50:03.574826 13911 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJN9WRF/build/release/bin/kudu as pid 27105
I20260924 13:50:03.576581 26971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:50:03.566570 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.149.195:48055 (local address 127.13.149.254:34283)
0924 13:50:03.566643 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0924 13:50:03.566646 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 13:50:03.566818 (+ 172us) server_negotiation.cc:407] Connection header received
0924 13:50:03.567325 (+ 507us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:50:03.567327 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:50:03.567339 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:50:03.567424 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:50:03.567930 (+ 506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:50:03.568693 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:50:03.569722 (+ 1029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:50:03.569996 (+ 274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:50:03.570076 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:50:03.573238 (+ 3162us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:50:03.573240 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:50:03.573240 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 13:50:03.573248 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:50:03.574915 (+ 1667us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:50:03.575475 (+ 560us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:50:03.575477 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:50:03.575477 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:50:03.575539 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:50:03.575857 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:50:03.575858 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:50:03.575858 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:50:03.576025 (+ 167us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:50:03.576089 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0924 13:50:03.576418 (+ 329us) server_negotiation.cc:299] Negotiation successful
0924 13:50:03.576493 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260924 13:50:03.576774 27238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:50:03.566544 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:34283 (local address 127.13.149.195:48055)
0924 13:50:03.566697 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0924 13:50:03.566738 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0924 13:50:03.567219 (+ 481us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:50:03.567474 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:50:03.567477 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:50:03.567653 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0924 13:50:03.567842 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:50:03.567846 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:50:03.568784 (+ 938us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:50:03.568784 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:50:03.569637 (+ 853us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:50:03.569640 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:50:03.570087 (+ 447us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:50:03.570088 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:50:03.570179 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:50:03.571472 (+ 1293us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 13:50:03.571482 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0924 13:50:03.573137 (+ 1655us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 13:50:03.574990 (+ 1853us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 13:50:03.574998 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 13:50:03.575000 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 13:50:03.575401 (+ 401us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 13:50:03.575632 (+ 231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 13:50:03.575633 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 13:50:03.575634 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 13:50:03.575783 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 13:50:03.576117 (+ 334us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 13:50:03.576124 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 13:50:03.576291 (+ 167us) client_negotiation.cc:769] Sending connection context
0924 13:50:03.576395 (+ 104us) client_negotiation.cc:240] Negotiation successful
0924 13:50:03.576539 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":85,"threads_started":1}
I20260924 13:50:03.577533 27236 heartbeater.cc:347] Connected to a master server at 127.13.149.254:34283
I20260924 13:50:03.577661 27236 heartbeater.cc:464] Registering TS with master...
I20260924 13:50:03.577860 27236 heartbeater.cc:511] Master 127.13.149.254:34283 requested a full tablet report, sending...
W20260924 13:50:03.578517 27236 heartbeater.cc:644] Failed to heartbeat to 127.13.149.254:34283 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260924 13:50:03.578640 27236 heartbeater.cc:464] Registering TS with master...
I20260924 13:50:03.578730 27236 heartbeater.cc:511] Master 127.13.149.254:34283 requested a full tablet report, sending...
I20260924 13:50:03.579025 27236 heartbeater.cc:464] Registering TS with master...
I20260924 13:50:03.579103 27236 heartbeater.cc:511] Master 127.13.149.254:34283 requested a full tablet report, sending...
W20260924 13:50:03.579588 27236 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260924 13:50:03.589798 27239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:50:03.575204 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49826 (local address 127.13.149.254:34283)
0924 13:50:03.575317 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0924 13:50:03.575319 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:50:03.575570 (+ 251us) server_negotiation.cc:407] Connection header received
0924 13:50:03.575619 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:50:03.575620 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:50:03.575631 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:50:03.575690 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:50:03.576871 (+ 1181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:50:03.577632 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:50:03.578951 (+ 1319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:50:03.579257 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:50:03.579341 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:50:03.580830 (+ 1489us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:50:03.580832 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:50:03.580832 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 13:50:03.580838 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:50:03.582438 (+ 1600us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:50:03.588980 (+ 6542us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:50:03.588982 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:50:03.588982 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:50:03.589061 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:50:03.589246 (+ 185us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:50:03.589247 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:50:03.589247 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:50:03.589368 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:50:03.589557 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0924 13:50:03.589612 (+ 55us) server_negotiation.cc:299] Negotiation successful
0924 13:50:03.589696 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86,"thread_start_us":55,"threads_started":1}
Sep 24 13:50:04 dist-test-slave-sjl2 krb5kdc[25774](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790257804, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.149.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 13:50:04.216100 26968 heartbeater.cc:464] Registering TS with master...
I20260924 13:50:04.216295 26968 heartbeater.cc:511] Master 127.13.149.254:34283 requested a full tablet report, sending...
I20260924 13:50:04.216838 26770 ts_manager.cc:194] Registered new tserver with Master: b38a33372efe47a28d157e2781f2c069 (127.13.149.193:38631)
I20260924 13:50:04.217554 26770 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.149.193@KRBTEST.COM'} at 127.13.149.193:60573
I20260924 13:50:04.388527 27102 heartbeater.cc:464] Registering TS with master...
I20260924 13:50:04.388696 27102 heartbeater.cc:511] Master 127.13.149.254:34283 requested a full tablet report, sending...
I20260924 13:50:04.389057 26770 ts_manager.cc:194] Registered new tserver with Master: ba730d9d9a2d4f8b87161cec9bcee786 (127.13.149.194:33273)
I20260924 13:50:04.389597 26770 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.149.194@KRBTEST.COM'} at 127.13.149.194:36569
Sep 24 13:50:04 dist-test-slave-sjl2 krb5kdc[25774](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257804, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.149.254@KRBTEST.COM for HTTP/127.13.149.212@KRBTEST.COM
I20260924 13:50:04.579754 27236 heartbeater.cc:464] Registering TS with master...
I20260924 13:50:04.579911 27236 heartbeater.cc:511] Master 127.13.149.254:34283 requested a full tablet report, sending...
I20260924 13:50:04.580257 26770 ts_manager.cc:194] Registered new tserver with Master: a9eb7fbc08de4cc78709a9e6f9b7823d (127.13.149.195:46435)
I20260924 13:50:04.580739 26770 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.149.195@KRBTEST.COM'} at 127.13.149.195:48055
I20260924 13:50:04.584134 13911 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260924 13:50:04.596560 27257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:50:04.585052 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49842 (local address 127.13.149.254:34283)
0924 13:50:04.585637 (+ 585us) server_negotiation.cc:206] Beginning negotiation
0924 13:50:04.585639 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:50:04.589093 (+ 3454us) server_negotiation.cc:407] Connection header received
0924 13:50:04.589196 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:50:04.589197 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:50:04.589210 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:50:04.589251 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:50:04.589965 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:50:04.590763 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:50:04.591785 (+ 1022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:50:04.592113 (+ 328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:50:04.592152 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:50:04.593691 (+ 1539us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:50:04.593692 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:50:04.593693 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 13:50:04.593700 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:50:04.595354 (+ 1654us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:50:04.595746 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:50:04.595747 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:50:04.595748 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:50:04.595805 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:50:04.595979 (+ 174us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:50:04.595979 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:50:04.595980 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:50:04.596100 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:50:04.596171 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0924 13:50:04.596372 (+ 201us) server_negotiation.cc:299] Negotiation successful
0924 13:50:04.596453 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":560,"thread_start_us":59,"threads_started":1}
I20260924 13:50:04.599041 26770 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49842:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 13:50:04.601480 26770 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 13:50:04.618345 27265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:50:04.604573 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.194:33273 (local address 127.0.0.1:59764)
0924 13:50:04.606113 (+ 1540us) negotiation.cc:107] Waiting for socket to connect
0924 13:50:04.606156 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0924 13:50:04.606230 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:50:04.607935 (+ 1705us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:50:04.607935 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:50:04.607937 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 13:50:04.607991 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:50:04.607992 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:50:04.613451 (+ 5459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:50:04.613453 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:50:04.614574 (+ 1121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:50:04.614577 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:50:04.617983 (+ 3406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:50:04.617985 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:50:04.618087 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:50:04.618104 (+ 17us) client_negotiation.cc:769] Sending connection context
0924 13:50:04.618179 (+ 75us) client_negotiation.cc:240] Negotiation successful
0924 13:50:04.618248 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1510,"thread_start_us":91,"threads_started":1}
I20260924 13:50:04.621308 27268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:50:04.604806 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.195:46435 (local address 127.0.0.1:50786)
0924 13:50:04.606958 (+ 2152us) negotiation.cc:107] Waiting for socket to connect
0924 13:50:04.606969 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0924 13:50:04.607025 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:50:04.607549 (+ 524us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:50:04.607552 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:50:04.607555 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 13:50:04.607621 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:50:04.607623 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:50:04.614909 (+ 7286us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:50:04.614910 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:50:04.616005 (+ 1095us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:50:04.616008 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:50:04.621046 (+ 5038us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:50:04.621048 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:50:04.621117 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:50:04.621121 (+ 4us) client_negotiation.cc:769] Sending connection context
0924 13:50:04.621169 (+ 48us) client_negotiation.cc:240] Negotiation successful
0924 13:50:04.621218 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2132,"thread_start_us":65,"threads_started":1}
I20260924 13:50:04.621747 27264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:50:04.605453 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.193:38631 (local address 127.0.0.1:39194)
0924 13:50:04.610233 (+ 4780us) negotiation.cc:107] Waiting for socket to connect
0924 13:50:04.610243 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 13:50:04.610288 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:50:04.610557 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:50:04.610558 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:50:04.610560 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 13:50:04.610610 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:50:04.610611 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:50:04.611800 (+ 1189us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:50:04.611801 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:50:04.618940 (+ 7139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:50:04.618943 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:50:04.621492 (+ 2549us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:50:04.621492 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:50:04.621560 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:50:04.621592 (+ 32us) client_negotiation.cc:769] Sending connection context
0924 13:50:04.621639 (+ 47us) client_negotiation.cc:240] Negotiation successful
0924 13:50:04.621681 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4764,"thread_start_us":56,"threads_started":1}
I20260924 13:50:04.622036 27269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:50:04.605628 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39194 (local address 127.13.149.193:38631)
0924 13:50:04.607126 (+ 1498us) server_negotiation.cc:206] Beginning negotiation
0924 13:50:04.607131 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 13:50:04.610344 (+ 3213us) server_negotiation.cc:407] Connection header received
0924 13:50:04.610384 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:50:04.610387 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:50:04.610402 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:50:04.610472 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 13:50:04.610695 (+ 223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:50:04.611704 (+ 1009us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:50:04.619041 (+ 7337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:50:04.619874 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:50:04.619954 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:50:04.620023 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0924 13:50:04.621900 (+ 1877us) server_negotiation.cc:299] Negotiation successful
0924 13:50:04.621957 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1452,"thread_start_us":58,"threads_started":1}
I20260924 13:50:04.622480 27267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:50:04.605272 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59764 (local address 127.13.149.194:33273)
0924 13:50:04.606418 (+ 1146us) server_negotiation.cc:206] Beginning negotiation
0924 13:50:04.606421 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 13:50:04.606430 (+ 9us) server_negotiation.cc:407] Connection header received
0924 13:50:04.606463 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:50:04.606465 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:50:04.607362 (+ 897us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:50:04.607450 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 13:50:04.608067 (+ 617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:50:04.613365 (+ 5298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:50:04.616984 (+ 3619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:50:04.617812 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:50:04.617885 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:50:04.617949 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0924 13:50:04.622342 (+ 4393us) server_negotiation.cc:299] Negotiation successful
0924 13:50:04.622405 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1117,"thread_start_us":48,"threads_started":1}
I20260924 13:50:04.622918 27266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:50:04.605093 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50786 (local address 127.13.149.195:46435)
0924 13:50:04.606351 (+ 1258us) server_negotiation.cc:206] Beginning negotiation
0924 13:50:04.606356 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 13:50:04.607156 (+ 800us) server_negotiation.cc:407] Connection header received
0924 13:50:04.607192 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:50:04.607195 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:50:04.607210 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:50:04.607277 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 13:50:04.608268 (+ 991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:50:04.614830 (+ 6562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:50:04.616405 (+ 1575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:50:04.620837 (+ 4432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:50:04.620952 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:50:04.621010 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0924 13:50:04.622765 (+ 1755us) server_negotiation.cc:299] Negotiation successful
0924 13:50:04.622845 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1225,"thread_start_us":46,"threads_started":1}
I20260924 13:50:04.622766 26902 tablet_service.cc:1467] Processing CreateTablet for tablet 322fe00a1a72491da32b3f88e7ce5efa (DEFAULT_TABLE table=test-table [id=8921d6804a654293818397e2e9526c56]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 13:50:04.623201 26902 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 322fe00a1a72491da32b3f88e7ce5efa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 13:50:04.623178 27033 tablet_service.cc:1467] Processing CreateTablet for tablet 322fe00a1a72491da32b3f88e7ce5efa (DEFAULT_TABLE table=test-table [id=8921d6804a654293818397e2e9526c56]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 13:50:04.623523 27033 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 322fe00a1a72491da32b3f88e7ce5efa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 13:50:04.623507 27170 tablet_service.cc:1467] Processing CreateTablet for tablet 322fe00a1a72491da32b3f88e7ce5efa (DEFAULT_TABLE table=test-table [id=8921d6804a654293818397e2e9526c56]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 13:50:04.623805 27170 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 322fe00a1a72491da32b3f88e7ce5efa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 13:50:04.626242 27270 tablet_bootstrap.cc:492] T 322fe00a1a72491da32b3f88e7ce5efa P b38a33372efe47a28d157e2781f2c069: Bootstrap starting.
I20260924 13:50:04.627229 27270 tablet_bootstrap.cc:654] T 322fe00a1a72491da32b3f88e7ce5efa P b38a33372efe47a28d157e2781f2c069: Neither blocks nor log segments found. Creating new log.
I20260924 13:50:04.627591 27270 log.cc:824] T 322fe00a1a72491da32b3f88e7ce5efa P b38a33372efe47a28d157e2781f2c069: Log is configured to *not* fsync() on all Append() calls
I20260924 13:50:04.627929 27271 tablet_bootstrap.cc:492] T 322fe00a1a72491da32b3f88e7ce5efa P ba730d9d9a2d4f8b87161cec9bcee786: Bootstrap starting.
I20260924 13:50:04.628685 27271 tablet_bootstrap.cc:654] T 322fe00a1a72491da32b3f88e7ce5efa P ba730d9d9a2d4f8b87161cec9bcee786: Neither blocks nor log segments found. Creating new log.
I20260924 13:50:04.629608 27270 tablet_bootstrap.cc:492] T 322fe00a1a72491da32b3f88e7ce5efa P b38a33372efe47a28d157e2781f2c069: No bootstrap required, opened a new log
I20260924 13:50:04.629699 27270 ts_tablet_manager.cc:1397] T 322fe00a1a72491da32b3f88e7ce5efa P b38a33372efe47a28d157e2781f2c069: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260924 13:50:04.633056 27270 raft_consensus.cc:348] T 322fe00a1a72491da32b3f88e7ce5efa P b38a33372efe47a28d157e2781f2c069 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38a33372efe47a28d157e2781f2c069" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 38631 } } peers { permanent_uuid: "ba730d9d9a2d4f8b87161cec9bcee786" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 33273 } } peers { permanent_uuid: "a9eb7fbc08de4cc78709a9e6f9b7823d" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 46435 } }
I20260924 13:50:04.633277 27270 raft_consensus.cc:374] T 322fe00a1a72491da32b3f88e7ce5efa P b38a33372efe47a28d157e2781f2c069 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 13:50:04.633322 27270 raft_consensus.cc:729] T 322fe00a1a72491da32b3f88e7ce5efa P b38a33372efe47a28d157e2781f2c069 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b38a33372efe47a28d157e2781f2c069, State: Initialized, Role: FOLLOWER
I20260924 13:50:04.633467 27270 consensus_queue.cc:260] T 322fe00a1a72491da32b3f88e7ce5efa P b38a33372efe47a28d157e2781f2c069 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38a33372efe47a28d157e2781f2c069" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 38631 } } peers { permanent_uuid: "ba730d9d9a2d4f8b87161cec9bcee786" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 33273 } } peers { permanent_uuid: "a9eb7fbc08de4cc78709a9e6f9b7823d" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 46435 } }
I20260924 13:50:04.633777 27270 ts_tablet_manager.cc:1428] T 322fe00a1a72491da32b3f88e7ce5efa P b38a33372efe47a28d157e2781f2c069: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260924 13:50:04.633906 26968 heartbeater.cc:503] Master 127.13.149.254:34283 was elected leader, sending a full tablet report...
I20260924 13:50:04.634819 27271 log.cc:824] T 322fe00a1a72491da32b3f88e7ce5efa P ba730d9d9a2d4f8b87161cec9bcee786: Log is configured to *not* fsync() on all Append() calls
I20260924 13:50:04.635270 27272 tablet_bootstrap.cc:492] T 322fe00a1a72491da32b3f88e7ce5efa P a9eb7fbc08de4cc78709a9e6f9b7823d: Bootstrap starting.
I20260924 13:50:04.636039 27272 tablet_bootstrap.cc:654] T 322fe00a1a72491da32b3f88e7ce5efa P a9eb7fbc08de4cc78709a9e6f9b7823d: Neither blocks nor log segments found. Creating new log.
I20260924 13:50:04.636535 27272 log.cc:824] T 322fe00a1a72491da32b3f88e7ce5efa P a9eb7fbc08de4cc78709a9e6f9b7823d: Log is configured to *not* fsync() on all Append() calls
I20260924 13:50:04.638183 27271 tablet_bootstrap.cc:492] T 322fe00a1a72491da32b3f88e7ce5efa P ba730d9d9a2d4f8b87161cec9bcee786: No bootstrap required, opened a new log
I20260924 13:50:04.638276 27271 ts_tablet_manager.cc:1397] T 322fe00a1a72491da32b3f88e7ce5efa P ba730d9d9a2d4f8b87161cec9bcee786: Time spent bootstrapping tablet: real 0.010s user 0.001s sys 0.000s
I20260924 13:50:04.641527 27271 raft_consensus.cc:348] T 322fe00a1a72491da32b3f88e7ce5efa P ba730d9d9a2d4f8b87161cec9bcee786 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38a33372efe47a28d157e2781f2c069" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 38631 } } peers { permanent_uuid: "ba730d9d9a2d4f8b87161cec9bcee786" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 33273 } } peers { permanent_uuid: "a9eb7fbc08de4cc78709a9e6f9b7823d" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 46435 } }
I20260924 13:50:04.641741 27271 raft_consensus.cc:374] T 322fe00a1a72491da32b3f88e7ce5efa P ba730d9d9a2d4f8b87161cec9bcee786 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 13:50:04.641782 27271 raft_consensus.cc:729] T 322fe00a1a72491da32b3f88e7ce5efa P ba730d9d9a2d4f8b87161cec9bcee786 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ba730d9d9a2d4f8b87161cec9bcee786, State: Initialized, Role: FOLLOWER
I20260924 13:50:04.641935 27271 consensus_queue.cc:260] T 322fe00a1a72491da32b3f88e7ce5efa P ba730d9d9a2d4f8b87161cec9bcee786 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38a33372efe47a28d157e2781f2c069" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 38631 } } peers { permanent_uuid: "ba730d9d9a2d4f8b87161cec9bcee786" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 33273 } } peers { permanent_uuid: "a9eb7fbc08de4cc78709a9e6f9b7823d" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 46435 } }
I20260924 13:50:04.642232 27271 ts_tablet_manager.cc:1428] T 322fe00a1a72491da32b3f88e7ce5efa P ba730d9d9a2d4f8b87161cec9bcee786: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260924 13:50:04.642369 27102 heartbeater.cc:503] Master 127.13.149.254:34283 was elected leader, sending a full tablet report...
I20260924 13:50:04.643756 27272 tablet_bootstrap.cc:492] T 322fe00a1a72491da32b3f88e7ce5efa P a9eb7fbc08de4cc78709a9e6f9b7823d: No bootstrap required, opened a new log
I20260924 13:50:04.643844 27272 ts_tablet_manager.cc:1397] T 322fe00a1a72491da32b3f88e7ce5efa P a9eb7fbc08de4cc78709a9e6f9b7823d: Time spent bootstrapping tablet: real 0.009s user 0.001s sys 0.000s
I20260924 13:50:04.651355 27272 raft_consensus.cc:348] T 322fe00a1a72491da32b3f88e7ce5efa P a9eb7fbc08de4cc78709a9e6f9b7823d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38a33372efe47a28d157e2781f2c069" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 38631 } } peers { permanent_uuid: "ba730d9d9a2d4f8b87161cec9bcee786" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 33273 } } peers { permanent_uuid: "a9eb7fbc08de4cc78709a9e6f9b7823d" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 46435 } }
I20260924 13:50:04.651680 27272 raft_consensus.cc:374] T 322fe00a1a72491da32b3f88e7ce5efa P a9eb7fbc08de4cc78709a9e6f9b7823d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 13:50:04.651767 27272 raft_consensus.cc:729] T 322fe00a1a72491da32b3f88e7ce5efa P a9eb7fbc08de4cc78709a9e6f9b7823d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a9eb7fbc08de4cc78709a9e6f9b7823d, State: Initialized, Role: FOLLOWER
I20260924 13:50:04.651968 27272 consensus_queue.cc:260] T 322fe00a1a72491da32b3f88e7ce5efa P a9eb7fbc08de4cc78709a9e6f9b7823d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38a33372efe47a28d157e2781f2c069" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 38631 } } peers { permanent_uuid: "ba730d9d9a2d4f8b87161cec9bcee786" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 33273 } } peers { permanent_uuid: "a9eb7fbc08de4cc78709a9e6f9b7823d" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 46435 } }
I20260924 13:50:04.652348 27272 ts_tablet_manager.cc:1428] T 322fe00a1a72491da32b3f88e7ce5efa P a9eb7fbc08de4cc78709a9e6f9b7823d: Time spent starting tablet: real 0.008s user 0.004s sys 0.000s
I20260924 13:50:04.652509 27236 heartbeater.cc:503] Master 127.13.149.254:34283 was elected leader, sending a full tablet report...
I20260924 13:50:04.657331 27277 raft_consensus.cc:482] T 322fe00a1a72491da32b3f88e7ce5efa P ba730d9d9a2d4f8b87161cec9bcee786 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 13:50:04.657534 27277 raft_consensus.cc:504] T 322fe00a1a72491da32b3f88e7ce5efa P ba730d9d9a2d4f8b87161cec9bcee786 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38a33372efe47a28d157e2781f2c069" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 38631 } } peers { permanent_uuid: "ba730d9d9a2d4f8b87161cec9bcee786" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 33273 } } peers { permanent_uuid: "a9eb7fbc08de4cc78709a9e6f9b7823d" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 46435 } }
I20260924 13:50:04.658465 27277 leader_election.cc:288] T 322fe00a1a72491da32b3f88e7ce5efa P ba730d9d9a2d4f8b87161cec9bcee786 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b38a33372efe47a28d157e2781f2c069 (127.13.149.193:38631), a9eb7fbc08de4cc78709a9e6f9b7823d (127.13.149.195:46435)
I20260924 13:50:04.662329 27269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:50:04.658396 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.149.194:36223 (local address 127.13.149.193:38631)
0924 13:50:04.658447 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0924 13:50:04.658449 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:50:04.658698 (+ 249us) server_negotiation.cc:407] Connection header received
0924 13:50:04.658800 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:50:04.658802 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:50:04.658811 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:50:04.658840 (+ 29us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 13:50:04.659067 (+ 227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:50:04.659823 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:50:04.661384 (+ 1561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:50:04.661912 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:50:04.661958 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:50:04.661968 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0924 13:50:04.662211 (+ 243us) server_negotiation.cc:299] Negotiation successful
0924 13:50:04.662258 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260924 13:50:04.662768 27279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:50:04.658189 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.193:38631 (local address 127.13.149.194:36223)
0924 13:50:04.658635 (+ 446us) negotiation.cc:107] Waiting for socket to connect
0924 13:50:04.658647 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0924 13:50:04.658728 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:50:04.658910 (+ 182us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:50:04.658911 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:50:04.658913 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 13:50:04.658991 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:50:04.658993 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:50:04.659926 (+ 933us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:50:04.659927 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:50:04.661284 (+ 1357us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:50:04.661287 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:50:04.662051 (+ 764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:50:04.662052 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:50:04.662121 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:50:04.662125 (+ 4us) client_negotiation.cc:769] Sending connection context
0924 13:50:04.662621 (+ 496us) client_negotiation.cc:240] Negotiation successful
0924 13:50:04.662669 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":428,"thread_start_us":98,"threads_started":1}
I20260924 13:50:04.662843 27266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:50:04.658793 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.149.194:52247 (local address 127.13.149.195:46435)
0924 13:50:04.658830 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0924 13:50:04.658832 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:50:04.659463 (+ 631us) server_negotiation.cc:407] Connection header received
0924 13:50:04.659555 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:50:04.659556 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:50:04.659565 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:50:04.659590 (+ 25us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 13:50:04.659800 (+ 210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:50:04.660491 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:50:04.661895 (+ 1404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:50:04.662448 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:50:04.662491 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:50:04.662497 (+ 6us) server_negotiation.cc:1035] Waiting for connection context
0924 13:50:04.662719 (+ 222us) server_negotiation.cc:299] Negotiation successful
0924 13:50:04.662756 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260924 13:50:04.663064 27280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:50:04.658566 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.195:46435 (local address 127.13.149.194:52247)
0924 13:50:04.659389 (+ 823us) negotiation.cc:107] Waiting for socket to connect
0924 13:50:04.659400 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0924 13:50:04.659489 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:50:04.659672 (+ 183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:50:04.659673 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:50:04.659674 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 13:50:04.659736 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:50:04.659737 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:50:04.660577 (+ 840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:50:04.660577 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:50:04.661789 (+ 1212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:50:04.661792 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:50:04.662552 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:50:04.662553 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:50:04.662642 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:50:04.662649 (+ 7us) client_negotiation.cc:769] Sending connection context
0924 13:50:04.662952 (+ 303us) client_negotiation.cc:240] Negotiation successful
0924 13:50:04.662991 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":803,"thread_start_us":107,"threads_started":1}
I20260924 13:50:04.663349 27187 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "322fe00a1a72491da32b3f88e7ce5efa" candidate_uuid: "ba730d9d9a2d4f8b87161cec9bcee786" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a9eb7fbc08de4cc78709a9e6f9b7823d" is_pre_election: true
I20260924 13:50:04.663334 26922 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "322fe00a1a72491da32b3f88e7ce5efa" candidate_uuid: "ba730d9d9a2d4f8b87161cec9bcee786" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b38a33372efe47a28d157e2781f2c069" is_pre_election: true
I20260924 13:50:04.663488 27187 raft_consensus.cc:2454] T 322fe00a1a72491da32b3f88e7ce5efa P a9eb7fbc08de4cc78709a9e6f9b7823d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba730d9d9a2d4f8b87161cec9bcee786 in term 0.
I20260924 13:50:04.663489 26922 raft_consensus.cc:2454] T 322fe00a1a72491da32b3f88e7ce5efa P b38a33372efe47a28d157e2781f2c069 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba730d9d9a2d4f8b87161cec9bcee786 in term 0.
I20260924 13:50:04.663697 26991 leader_election.cc:302] T 322fe00a1a72491da32b3f88e7ce5efa P ba730d9d9a2d4f8b87161cec9bcee786 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a9eb7fbc08de4cc78709a9e6f9b7823d, ba730d9d9a2d4f8b87161cec9bcee786; no voters:
I20260924 13:50:04.663863 27277 raft_consensus.cc:2781] T 322fe00a1a72491da32b3f88e7ce5efa P ba730d9d9a2d4f8b87161cec9bcee786 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 13:50:04.663904 27277 raft_consensus.cc:482] T 322fe00a1a72491da32b3f88e7ce5efa P ba730d9d9a2d4f8b87161cec9bcee786 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 13:50:04.663924 27277 raft_consensus.cc:3037] T 322fe00a1a72491da32b3f88e7ce5efa P ba730d9d9a2d4f8b87161cec9bcee786 [term 0 FOLLOWER]: Advancing to term 1
I20260924 13:50:04.664742 27277 raft_consensus.cc:504] T 322fe00a1a72491da32b3f88e7ce5efa P ba730d9d9a2d4f8b87161cec9bcee786 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38a33372efe47a28d157e2781f2c069" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 38631 } } peers { permanent_uuid: "ba730d9d9a2d4f8b87161cec9bcee786" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 33273 } } peers { permanent_uuid: "a9eb7fbc08de4cc78709a9e6f9b7823d" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 46435 } }
I20260924 13:50:04.664875 27277 leader_election.cc:288] T 322fe00a1a72491da32b3f88e7ce5efa P ba730d9d9a2d4f8b87161cec9bcee786 [CANDIDATE]: Term 1 election: Requested vote from peers b38a33372efe47a28d157e2781f2c069 (127.13.149.193:38631), a9eb7fbc08de4cc78709a9e6f9b7823d (127.13.149.195:46435)
I20260924 13:50:04.665069 26922 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "322fe00a1a72491da32b3f88e7ce5efa" candidate_uuid: "ba730d9d9a2d4f8b87161cec9bcee786" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b38a33372efe47a28d157e2781f2c069"
I20260924 13:50:04.665099 27187 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "322fe00a1a72491da32b3f88e7ce5efa" candidate_uuid: "ba730d9d9a2d4f8b87161cec9bcee786" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a9eb7fbc08de4cc78709a9e6f9b7823d"
I20260924 13:50:04.665153 26922 raft_consensus.cc:3037] T 322fe00a1a72491da32b3f88e7ce5efa P b38a33372efe47a28d157e2781f2c069 [term 0 FOLLOWER]: Advancing to term 1
I20260924 13:50:04.665169 27187 raft_consensus.cc:3037] T 322fe00a1a72491da32b3f88e7ce5efa P a9eb7fbc08de4cc78709a9e6f9b7823d [term 0 FOLLOWER]: Advancing to term 1
I20260924 13:50:04.665940 27187 raft_consensus.cc:2454] T 322fe00a1a72491da32b3f88e7ce5efa P a9eb7fbc08de4cc78709a9e6f9b7823d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba730d9d9a2d4f8b87161cec9bcee786 in term 1.
I20260924 13:50:04.665941 26922 raft_consensus.cc:2454] T 322fe00a1a72491da32b3f88e7ce5efa P b38a33372efe47a28d157e2781f2c069 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba730d9d9a2d4f8b87161cec9bcee786 in term 1.
I20260924 13:50:04.666093 26991 leader_election.cc:302] T 322fe00a1a72491da32b3f88e7ce5efa P ba730d9d9a2d4f8b87161cec9bcee786 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a9eb7fbc08de4cc78709a9e6f9b7823d, ba730d9d9a2d4f8b87161cec9bcee786; no voters:
I20260924 13:50:04.666182 27277 raft_consensus.cc:2781] T 322fe00a1a72491da32b3f88e7ce5efa P ba730d9d9a2d4f8b87161cec9bcee786 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 13:50:04.666318 27277 raft_consensus.cc:686] T 322fe00a1a72491da32b3f88e7ce5efa P ba730d9d9a2d4f8b87161cec9bcee786 [term 1 LEADER]: Becoming Leader. State: Replica: ba730d9d9a2d4f8b87161cec9bcee786, State: Running, Role: LEADER
I20260924 13:50:04.666407 27277 consensus_queue.cc:237] T 322fe00a1a72491da32b3f88e7ce5efa P ba730d9d9a2d4f8b87161cec9bcee786 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38a33372efe47a28d157e2781f2c069" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 38631 } } peers { permanent_uuid: "ba730d9d9a2d4f8b87161cec9bcee786" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 33273 } } peers { permanent_uuid: "a9eb7fbc08de4cc78709a9e6f9b7823d" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 46435 } }
I20260924 13:50:04.667208 26769 catalog_manager.cc:5495] T 322fe00a1a72491da32b3f88e7ce5efa P ba730d9d9a2d4f8b87161cec9bcee786 reported cstate change: term changed from 0 to 1, leader changed from <none> to ba730d9d9a2d4f8b87161cec9bcee786 (127.13.149.194). New cstate: current_term: 1 leader_uuid: "ba730d9d9a2d4f8b87161cec9bcee786" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38a33372efe47a28d157e2781f2c069" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 38631 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba730d9d9a2d4f8b87161cec9bcee786" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 33273 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a9eb7fbc08de4cc78709a9e6f9b7823d" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 46435 } health_report { overall_health: UNKNOWN } } }
I20260924 13:50:04.684813 27267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:50:04.681665 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59776 (local address 127.13.149.194:33273)
0924 13:50:04.681706 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0924 13:50:04.681708 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:50:04.681717 (+ 9us) server_negotiation.cc:407] Connection header received
0924 13:50:04.681762 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:50:04.681771 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:50:04.681794 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:50:04.681876 (+ 82us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 13:50:04.682014 (+ 138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:50:04.682739 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:50:04.683850 (+ 1111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:50:04.684172 (+ 322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:50:04.684249 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:50:04.684437 (+ 188us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 13:50:04.684576 (+ 139us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 13:50:04.684654 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0924 13:50:04.684707 (+ 53us) server_negotiation.cc:299] Negotiation successful
0924 13:50:04.684729 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260924 13:50:04.686992 26922 raft_consensus.cc:1261] T 322fe00a1a72491da32b3f88e7ce5efa P b38a33372efe47a28d157e2781f2c069 [term 1 FOLLOWER]: Refusing update from remote peer ba730d9d9a2d4f8b87161cec9bcee786: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 13:50:04.687415 27187 raft_consensus.cc:1261] T 322fe00a1a72491da32b3f88e7ce5efa P a9eb7fbc08de4cc78709a9e6f9b7823d [term 1 FOLLOWER]: Refusing update from remote peer ba730d9d9a2d4f8b87161cec9bcee786: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 13:50:04.687636 27277 consensus_queue.cc:1035] T 322fe00a1a72491da32b3f88e7ce5efa P ba730d9d9a2d4f8b87161cec9bcee786 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b38a33372efe47a28d157e2781f2c069" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 38631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 13:50:04.687879 27281 consensus_queue.cc:1035] T 322fe00a1a72491da32b3f88e7ce5efa P ba730d9d9a2d4f8b87161cec9bcee786 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a9eb7fbc08de4cc78709a9e6f9b7823d" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 46435 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 13:50:04.691352 27284 mvcc.cc:205] Tried to move back new op lower bound from 7332895967994732544 to 7332895967914078208. Current Snapshot: MvccSnapshot[applied={T|T < 7332895967994732544}]
I20260924 13:50:04.692076 27287 mvcc.cc:205] Tried to move back new op lower bound from 7332895967994732544 to 7332895967914078208. Current Snapshot: MvccSnapshot[applied={T|T < 7332895967994732544}]
I20260924 13:50:04.694804 26770 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49842:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 13:50:04.694909 26770 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49842:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 13:50:04.695441 27285 mvcc.cc:205] Tried to move back new op lower bound from 7332895967994732544 to 7332895967914078208. Current Snapshot: MvccSnapshot[applied={T|T < 7332895967994732544}]
I20260924 13:50:04.695657 26770 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9bb72cc199554d53af2e97d01120dbd8: Sending DeleteTablet for 3 replicas of tablet 322fe00a1a72491da32b3f88e7ce5efa
I20260924 13:50:04.696151 27033 tablet_service.cc:1514] Processing DeleteTablet for tablet 322fe00a1a72491da32b3f88e7ce5efa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 13:50:04 UTC) from {username='kudu', principal='kudu/127.13.149.254@KRBTEST.COM'} at 127.0.0.1:59764
I20260924 13:50:04.696378 27170 tablet_service.cc:1514] Processing DeleteTablet for tablet 322fe00a1a72491da32b3f88e7ce5efa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 13:50:04 UTC) from {username='kudu', principal='kudu/127.13.149.254@KRBTEST.COM'} at 127.0.0.1:50786
I20260924 13:50:04.696689 26902 tablet_service.cc:1514] Processing DeleteTablet for tablet 322fe00a1a72491da32b3f88e7ce5efa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 13:50:04 UTC) from {username='kudu', principal='kudu/127.13.149.254@KRBTEST.COM'} at 127.0.0.1:39194
I20260924 13:50:04.697077 13911 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJN9WRF/build/release/bin/kudu with pid 26808
I20260924 13:50:04.697539 27293 tablet_replica.cc:330] T 322fe00a1a72491da32b3f88e7ce5efa P ba730d9d9a2d4f8b87161cec9bcee786: stopping tablet replica
I20260924 13:50:04.697628 27293 raft_consensus.cc:2229] T 322fe00a1a72491da32b3f88e7ce5efa P ba730d9d9a2d4f8b87161cec9bcee786 [term 1 LEADER]: Raft consensus shutting down.
I20260924 13:50:04.697702 27293 raft_consensus.cc:2258] T 322fe00a1a72491da32b3f88e7ce5efa P ba730d9d9a2d4f8b87161cec9bcee786 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 13:50:04.697842 27294 tablet_replica.cc:330] T 322fe00a1a72491da32b3f88e7ce5efa P a9eb7fbc08de4cc78709a9e6f9b7823d: stopping tablet replica
I20260924 13:50:04.697922 27294 raft_consensus.cc:2229] T 322fe00a1a72491da32b3f88e7ce5efa P a9eb7fbc08de4cc78709a9e6f9b7823d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 13:50:04.698006 27294 raft_consensus.cc:2258] T 322fe00a1a72491da32b3f88e7ce5efa P a9eb7fbc08de4cc78709a9e6f9b7823d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 13:50:04.698942 27293 ts_tablet_manager.cc:1905] T 322fe00a1a72491da32b3f88e7ce5efa P ba730d9d9a2d4f8b87161cec9bcee786: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 13:50:04.700803 27294 ts_tablet_manager.cc:1905] T 322fe00a1a72491da32b3f88e7ce5efa P a9eb7fbc08de4cc78709a9e6f9b7823d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 13:50:04.702262 27294 ts_tablet_manager.cc:1918] T 322fe00a1a72491da32b3f88e7ce5efa P a9eb7fbc08de4cc78709a9e6f9b7823d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 13:50:04.702345 27294 log.cc:1196] T 322fe00a1a72491da32b3f88e7ce5efa P a9eb7fbc08de4cc78709a9e6f9b7823d: Deleting WAL directory at /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790257629041645-13911-0/minicluster-data/ts-2/wal/wals/322fe00a1a72491da32b3f88e7ce5efa
I20260924 13:50:04.702512 27293 ts_tablet_manager.cc:1918] T 322fe00a1a72491da32b3f88e7ce5efa P ba730d9d9a2d4f8b87161cec9bcee786: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 13:50:04.702576 27294 ts_tablet_manager.cc:1939] T 322fe00a1a72491da32b3f88e7ce5efa P a9eb7fbc08de4cc78709a9e6f9b7823d: Deleting consensus metadata
I20260924 13:50:04.702683 27293 log.cc:1196] T 322fe00a1a72491da32b3f88e7ce5efa P ba730d9d9a2d4f8b87161cec9bcee786: Deleting WAL directory at /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790257629041645-13911-0/minicluster-data/ts-1/wal/wals/322fe00a1a72491da32b3f88e7ce5efa
I20260924 13:50:04.703011 27293 ts_tablet_manager.cc:1939] T 322fe00a1a72491da32b3f88e7ce5efa P ba730d9d9a2d4f8b87161cec9bcee786: Deleting consensus metadata
I20260924 13:50:04.703001 26756 catalog_manager.cc:4841] TS a9eb7fbc08de4cc78709a9e6f9b7823d (127.13.149.195:46435): tablet 322fe00a1a72491da32b3f88e7ce5efa (table test-table [id=8921d6804a654293818397e2e9526c56]) successfully deleted
I20260924 13:50:04.703428 26756 catalog_manager.cc:4841] TS ba730d9d9a2d4f8b87161cec9bcee786 (127.13.149.194:33273): tablet 322fe00a1a72491da32b3f88e7ce5efa (table test-table [id=8921d6804a654293818397e2e9526c56]) successfully deleted
W20260924 13:50:04.711901 26754 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.13.149.193:38631 (error 108)
I20260924 13:50:04.712561 13911 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJN9WRF/build/release/bin/kudu with pid 26972
I20260924 13:50:04.717548 27264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:50:04.717365 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.193:38631 (local address 127.0.0.1:39204)
0924 13:50:04.717439 (+ 74us) negotiation.cc:107] Waiting for socket to connect
0924 13:50:04.717500 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.149.193:38631: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260924 13:50:04.717710 26754 catalog_manager.cc:4584] TS b38a33372efe47a28d157e2781f2c069 (127.13.149.193:38631): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 322fe00a1a72491da32b3f88e7ce5efa: Network error: Client connection negotiation failed: client connection to 127.13.149.193:38631: connect: Connection refused (error 111)
I20260924 13:50:04.719252 13911 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJN9WRF/build/release/bin/kudu with pid 27105
I20260924 13:50:04.725061 13911 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJN9WRF/build/release/bin/kudu with pid 26737
I20260924 13:50:04.730296 13911 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260924 13:50:05.064862 13911 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260924 13:50:05.064988 13911 mini_ranger.cc:66] Stopping Ranger...
I20260924 13:50:05.399529 13911 mini_ranger.cc:68] Stopped Ranger
2026-09-24 13:50:05.399 UTC [25804] LOG: received smart shutdown request
2026-09-24 13:50:05.403 UTC [25804] LOG: background worker "logical replication launcher" (PID 25813) exited with exit code 1
2026-09-24 13:50:05.404 UTC [25808] LOG: shutting down
2026-09-24 13:50:05.577 UTC [25804] LOG: database system is shut down
2026-09-24T13:50:05Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (46771 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790257629041645-13911-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 13:50:05 dist-test-slave-sjl2 krb5kdc[27305](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 13:50:05 dist-test-slave-sjl2 krb5kdc[27305](info): set up 2 sockets
Sep 24 13:50:05 dist-test-slave-sjl2 krb5kdc[27305](info): commencing operation
krb5kdc: starting...
W20260924 13:50:06.609740 13911 mini_kdc.cc:121] Time spent starting KDC: real 0.930s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790257629041645-13911-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790257629041645-13911-0/krb5kdc/test-user.keytab.
Sep 24 13:50:06 dist-test-slave-sjl2 krb5kdc[27305](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257806, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T13:50:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T13:50:06Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.149.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.149.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.149.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.254.keytab.
Entry for principal kudu/127.13.149.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.254.keytab.
I20260924 13:50:06.731810 13911 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJN9WRF/build/release/bin/kudu
/tmp/dist-test-taskJN9WRF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790257629041645-13911-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790257629041645-13911-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790257629041645-13911-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790257629041645-13911-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.149.254:46275
--webserver_interface=127.13.149.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.254.keytab
--principal=kudu/127.13.149.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.149.212:45357
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.149.254:46275,127.13.149.253:43875,127.13.149.252:33119
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790257629041645-13911-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790257629041645-13911-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790257629041645-13911-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 13:50:06.815248 27318 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 13:50:06.815487 27318 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 13:50:06.815512 27318 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 13:50:06.816736 27318 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 13:50:06.816792 27318 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 13:50:06.816804 27318 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 13:50:06.816816 27318 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 13:50:06.816828 27318 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 13:50:06.818457 27318 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.149.212:45357
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790257629041645-13911-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790257629041645-13911-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.149.254:46275,127.13.149.253:43875,127.13.149.252:33119
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.254.keytab
--principal=kudu/127.13.149.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.149.254:46275
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790257629041645-13911-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.149.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27318
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790257629041645-13911-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 12:29:37 UTC on ccycloud.quasar-iqzbxn.root.comops.site
build id 4670106
I20260924 13:50:06.818729 27318 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 13:50:06.818965 27318 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 13:50:06.824005 27324 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 13:50:06.823983 27323 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 13:50:06.824024 27318 server_base.cc:1034] running on GCE node
W20260924 13:50:06.823989 27326 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 13:50:06.824476 27318 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 13:50:06.824750 27318 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 13:50:06.825896 27318 hybrid_clock.cc:648] HybridClock initialized: now 1790257806825878 us; error 33 us; skew 500 ppm
Sep 24 13:50:06 dist-test-slave-sjl2 krb5kdc[27305](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257806, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.149.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 13:50:06.832271 27318 init.cc:377] Logged in from keytab as kudu/127.13.149.254@KRBTEST.COM (short username kudu)
I20260924 13:50:06.832842 27318 webserver.cc:460] Webserver started at http://127.13.149.254:35579/ using document root <none> and password file <none>
I20260924 13:50:06.833076 27318 fs_manager.cc:359] Metadata directory not provided
I20260924 13:50:06.833124 27318 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 13:50:06.833251 27318 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 13:50:06.834308 27318 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790257629041645-13911-0/minicluster-data/master-0/data/instance:
uuid: "eef9f241ec1746809efe5a82656b136f"
format_stamp: "Formatted at 2026-09-24 13:50:06 on dist-test-slave-sjl2"
server_key: "f5506505676b1cde2f8ac2632d7e83c3"
server_key_iv: "1e478143dd67bbeb00680f678d5e111f"
server_key_version: "encryptionkey@0"
I20260924 13:50:06.834643 27318 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790257629041645-13911-0/minicluster-data/master-0/wal/instance:
uuid: "eef9f241ec1746809efe5a82656b136f"
format_stamp: "Formatted at 2026-09-24 13:50:06 on dist-test-slave-sjl2"
server_key: "f5506505676b1cde2f8ac2632d7e83c3"
server_key_iv: "1e478143dd67bbeb00680f678d5e111f"
server_key_version: "encryptionkey@0"
I20260924 13:50:06.835951 27318 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260924 13:50:06.836697 27333 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 13:50:06.836841 27318 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 13:50:06.836951 27318 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790257629041645-13911-0/minicluster-data/master-0/data,/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790257629041645-13911-0/minicluster-data/master-0/wal
uuid: "eef9f241ec1746809efe5a82656b136f"
format_stamp: "Formatted at 2026-09-24 13:50:06 on dist-test-slave-sjl2"
server_key: "f5506505676b1cde2f8ac2632d7e83c3"
server_key_iv: "1e478143dd67bbeb00680f678d5e111f"
server_key_version: "encryptionkey@0"
I20260924 13:50:06.837013 27318 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790257629041645-13911-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790257629041645-13911-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790257629041645-13911-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 13:50:06.856899 27318 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 13:50:06.859073 27318 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 13:50:06.859220 27318 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 13:50:06.863116 27318 rpc_server.cc:307] RPC server started. Bound to: 127.13.149.254:46275
I20260924 13:50:06.863154 27385 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.149.254:46275 every 8 connection(s)
I20260924 13:50:06.863476 27318 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790257629041645-13911-0/minicluster-data/master-0/data/info.pb
I20260924 13:50:06.864329 27386 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 13:50:06.865964 27386 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.149.254" port: 46275 } has no permanent_uuid. Determining permanent_uuid...
I20260924 13:50:06.867072 13911 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJN9WRF/build/release/bin/kudu as pid 27318
I20260924 13:50:06.867229 13911 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790257629041645-13911-0/minicluster-data/master-0/wal/instance
I20260924 13:50:06.867398 13911 external_mini_cluster.cc:1428] Setting key df7a4f2f4d4136f405a0e8490754a9e9
Sep 24 13:50:06 dist-test-slave-sjl2 krb5kdc[27305](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257806, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.149.254@KRBTEST.COM for kudu/127.13.149.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.13.149.253@KRBTEST.COM; defaulting to no policy
I20260924 13:50:06.876670 27388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:50:06.866622 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:46275 (local address 127.0.0.1:46418)
0924 13:50:06.866794 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0924 13:50:06.866837 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0924 13:50:06.867355 (+ 518us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:50:06.867698 (+ 343us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:50:06.867701 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:50:06.867962 (+ 261us) client_negotiation.cc:189] Negotiated authn=SASL
0924 13:50:06.868119 (+ 157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:50:06.868123 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:50:06.869372 (+ 1249us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:50:06.869373 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:50:06.870432 (+ 1059us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:50:06.870436 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:50:06.871007 (+ 571us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:50:06.871008 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:50:06.871090 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:50:06.871630 (+ 540us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 13:50:06.871645 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0924 13:50:06.873263 (+ 1618us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 13:50:06.875216 (+ 1953us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 13:50:06.875223 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 13:50:06.875225 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 13:50:06.875415 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 13:50:06.875653 (+ 238us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 13:50:06.875654 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 13:50:06.875654 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 13:50:06.875735 (+ 81us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 13:50:06.876166 (+ 431us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 13:50:06.876173 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 13:50:06.876235 (+ 62us) client_negotiation.cc:769] Sending connection context
0924 13:50:06.876329 (+ 94us) client_negotiation.cc:240] Negotiation successful
0924 13:50:06.876466 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":73,"threads_started":1}
I20260924 13:50:06.876681 27389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:50:06.866692 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46418 (local address 127.13.149.254:46275)
0924 13:50:06.866837 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0924 13:50:06.866844 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0924 13:50:06.866951 (+ 107us) server_negotiation.cc:407] Connection header received
0924 13:50:06.867512 (+ 561us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:50:06.867515 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:50:06.867538 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:50:06.867662 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:50:06.868211 (+ 549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:50:06.869282 (+ 1071us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:50:06.870540 (+ 1258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:50:06.870905 (+ 365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:50:06.871025 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:50:06.873358 (+ 2333us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:50:06.873367 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:50:06.873367 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 13:50:06.873373 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:50:06.875125 (+ 1752us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:50:06.875482 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:50:06.875484 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:50:06.875484 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:50:06.875565 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:50:06.875787 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:50:06.875788 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:50:06.875788 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:50:06.876013 (+ 225us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:50:06.876079 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0924 13:50:06.876442 (+ 363us) server_negotiation.cc:299] Negotiation successful
0924 13:50:06.876516 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":115,"thread_start_us":73,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.149.253@KRBTEST.COM" created.
I20260924 13:50:06.877488 27386 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.149.253" port: 43875 } has no permanent_uuid. Determining permanent_uuid...
I20260924 13:50:06.877851 27388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:50:06.877742 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.253:43875 (local address 127.0.0.1:59520)
0924 13:50:06.877787 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0924 13:50:06.877829 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.149.253:43875: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260924 13:50:06.877966 27337 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.13.149.253:43875: connect: Connection refused (error 111)
I20260924 13:50:06.878425 27388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:50:06.878340 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.253:43875 (local address 127.0.0.1:59528)
0924 13:50:06.878381 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0924 13:50:06.878405 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.149.253:43875: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260924 13:50:06.878510 27386 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.149.253:43875: Network error: Client connection negotiation failed: client connection to 127.13.149.253:43875: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.149.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.253.keytab.
Entry for principal kudu/127.13.149.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.253.keytab.
I20260924 13:50:06.893723 13911 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJN9WRF/build/release/bin/kudu
/tmp/dist-test-taskJN9WRF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790257629041645-13911-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790257629041645-13911-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790257629041645-13911-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790257629041645-13911-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.149.253:43875
--webserver_interface=127.13.149.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.253.keytab
--principal=kudu/127.13.149.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.149.212:45357
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.149.254:46275,127.13.149.253:43875,127.13.149.252:33119
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790257629041645-13911-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790257629041645-13911-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790257629041645-13911-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260924 13:50:06.927821 27386 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.149.253" port: 43875 } attempt: 1
I20260924 13:50:06.928369 27388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:50:06.928217 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.253:43875 (local address 127.0.0.1:59540)
0924 13:50:06.928275 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0924 13:50:06.928340 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.149.253:43875: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260924 13:50:06.928700 27388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:50:06.928618 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.253:43875 (local address 127.0.0.1:59550)
0924 13:50:06.928658 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0924 13:50:06.928683 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.149.253:43875: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260924 13:50:06.928812 27386 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.149.253:43875: Network error: Client connection negotiation failed: client connection to 127.13.149.253:43875: connect: Connection refused (error 111)
W20260924 13:50:06.975255 27393 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 13:50:06.975486 27393 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 13:50:06.975520 27393 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 13:50:06.976728 27393 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 13:50:06.976776 27393 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 13:50:06.976795 27393 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 13:50:06.976811 27393 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 13:50:06.976827 27393 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 13:50:06.978479 27393 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.149.212:45357
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790257629041645-13911-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790257629041645-13911-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.13.149.254:46275,127.13.149.253:43875,127.13.149.252:33119
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.253.keytab
--principal=kudu/127.13.149.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.149.253:43875
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790257629041645-13911-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.13.149.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27393
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790257629041645-13911-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 12:29:37 UTC on ccycloud.quasar-iqzbxn.root.comops.site
build id 4670106
I20260924 13:50:06.978731 27393 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 13:50:06.978950 27393 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 13:50:06.983902 27399 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 13:50:06.983992 27401 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 13:50:06.983906 27398 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 13:50:06.984139 27393 server_base.cc:1034] running on GCE node
I20260924 13:50:06.984429 27393 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 13:50:06.984694 27393 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 13:50:06.985828 27393 hybrid_clock.cc:648] HybridClock initialized: now 1790257806985815 us; error 27 us; skew 500 ppm
Sep 24 13:50:06 dist-test-slave-sjl2 krb5kdc[27305](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257806, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.149.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 13:50:06.988449 27393 init.cc:377] Logged in from keytab as kudu/127.13.149.253@KRBTEST.COM (short username kudu)
I20260924 13:50:06.988977 27393 webserver.cc:460] Webserver started at http://127.13.149.253:43879/ using document root <none> and password file <none>
I20260924 13:50:06.989221 27393 fs_manager.cc:359] Metadata directory not provided
I20260924 13:50:06.989272 27393 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 13:50:06.989399 27393 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 13:50:06.990487 27393 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790257629041645-13911-0/minicluster-data/master-1/data/instance:
uuid: "25e330f2c1e8482f9eb2a0cb3837e54a"
format_stamp: "Formatted at 2026-09-24 13:50:06 on dist-test-slave-sjl2"
server_key: "deda5f9ad0c2275acfbb9bb276cdd288"
server_key_iv: "59f3618e219e2d3e476edd975f68c8df"
server_key_version: "encryptionkey@0"
I20260924 13:50:06.990810 27393 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790257629041645-13911-0/minicluster-data/master-1/wal/instance:
uuid: "25e330f2c1e8482f9eb2a0cb3837e54a"
format_stamp: "Formatted at 2026-09-24 13:50:06 on dist-test-slave-sjl2"
server_key: "deda5f9ad0c2275acfbb9bb276cdd288"
server_key_iv: "59f3618e219e2d3e476edd975f68c8df"
server_key_version: "encryptionkey@0"
I20260924 13:50:06.992110 27393 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260924 13:50:06.992992 27408 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 13:50:06.993191 27393 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260924 13:50:06.993285 27393 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790257629041645-13911-0/minicluster-data/master-1/data,/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790257629041645-13911-0/minicluster-data/master-1/wal
uuid: "25e330f2c1e8482f9eb2a0cb3837e54a"
format_stamp: "Formatted at 2026-09-24 13:50:06 on dist-test-slave-sjl2"
server_key: "deda5f9ad0c2275acfbb9bb276cdd288"
server_key_iv: "59f3618e219e2d3e476edd975f68c8df"
server_key_version: "encryptionkey@0"
I20260924 13:50:06.993355 27393 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790257629041645-13911-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790257629041645-13911-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790257629041645-13911-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 13:50:06.996955 27386 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.149.253" port: 43875 } attempt: 2
I20260924 13:50:06.997392 27388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:50:06.997271 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.253:43875 (local address 127.0.0.1:59562)
0924 13:50:06.997330 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0924 13:50:06.997362 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.149.253:43875: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260924 13:50:06.997820 27388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:50:06.997717 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.253:43875 (local address 127.0.0.1:59564)
0924 13:50:06.997763 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0924 13:50:06.997792 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.149.253:43875: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260924 13:50:06.997917 27386 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.149.253:43875: Network error: Client connection negotiation failed: client connection to 127.13.149.253:43875: connect: Connection refused (error 111)
I20260924 13:50:07.004463 27393 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 13:50:07.006418 27393 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 13:50:07.006568 27393 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 13:50:07.010630 27393 rpc_server.cc:307] RPC server started. Bound to: 127.13.149.253:43875
I20260924 13:50:07.010666 27460 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.149.253:43875 every 8 connection(s)
I20260924 13:50:07.011006 27393 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790257629041645-13911-0/minicluster-data/master-1/data/info.pb
I20260924 13:50:07.011885 27461 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 13:50:07.013499 27461 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.149.254" port: 46275 } has no permanent_uuid. Determining permanent_uuid...
I20260924 13:50:07.018778 13911 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJN9WRF/build/release/bin/kudu as pid 27393
I20260924 13:50:07.018890 13911 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790257629041645-13911-0/minicluster-data/master-1/wal/instance
I20260924 13:50:07.019080 13911 external_mini_cluster.cc:1428] Setting key f4f075b0fae80d70e591b1985ce7f8a2
Sep 24 13:50:07 dist-test-slave-sjl2 krb5kdc[27305](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257806, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.149.253@KRBTEST.COM for kudu/127.13.149.254@KRBTEST.COM
I20260924 13:50:07.023331 27389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:50:07.014196 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46422 (local address 127.13.149.254:46275)
0924 13:50:07.014322 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0924 13:50:07.014324 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:50:07.014576 (+ 252us) server_negotiation.cc:407] Connection header received
0924 13:50:07.014926 (+ 350us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:50:07.014928 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:50:07.014938 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:50:07.014998 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:50:07.015638 (+ 640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:50:07.016189 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:50:07.017278 (+ 1089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:50:07.017511 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:50:07.017584 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:50:07.019810 (+ 2226us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:50:07.019812 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:50:07.019812 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 13:50:07.019821 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:50:07.021611 (+ 1790us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:50:07.022186 (+ 575us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:50:07.022187 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:50:07.022188 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:50:07.022262 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:50:07.022592 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:50:07.022594 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:50:07.022594 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:50:07.022726 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:50:07.022805 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0924 13:50:07.023177 (+ 372us) server_negotiation.cc:299] Negotiation successful
0924 13:50:07.023260 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260924 13:50:07.023609 27463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:50:07.014171 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:46275 (local address 127.0.0.1:46422)
0924 13:50:07.014338 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0924 13:50:07.014375 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0924 13:50:07.014825 (+ 450us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:50:07.015058 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:50:07.015061 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:50:07.015334 (+ 273us) client_negotiation.cc:189] Negotiated authn=SASL
0924 13:50:07.015537 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:50:07.015542 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:50:07.016302 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:50:07.016303 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:50:07.017174 (+ 871us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:50:07.017177 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:50:07.017612 (+ 435us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:50:07.017613 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:50:07.017744 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:50:07.018264 (+ 520us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 13:50:07.018272 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0924 13:50:07.019709 (+ 1437us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 13:50:07.021714 (+ 2005us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 13:50:07.021725 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 13:50:07.021728 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0924 13:50:07.022058 (+ 330us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 13:50:07.022336 (+ 278us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 13:50:07.022337 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 13:50:07.022337 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 13:50:07.022496 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 13:50:07.022828 (+ 332us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 13:50:07.022837 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 13:50:07.023032 (+ 195us) client_negotiation.cc:769] Sending connection context
0924 13:50:07.023156 (+ 124us) client_negotiation.cc:240] Negotiation successful
0924 13:50:07.023352 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":74,"threads_started":1}
I20260924 13:50:07.024300 27461 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.149.253" port: 43875 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.13.149.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.149.252@KRBTEST.COM" created.
Sep 24 13:50:07 dist-test-slave-sjl2 krb5kdc[27305](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257806, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.149.253@KRBTEST.COM for kudu/127.13.149.253@KRBTEST.COM
I20260924 13:50:07.032863 27465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:50:07.024653 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59580 (local address 127.13.149.253:43875)
0924 13:50:07.024834 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0924 13:50:07.024840 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0924 13:50:07.024865 (+ 25us) server_negotiation.cc:407] Connection header received
0924 13:50:07.024889 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:50:07.024892 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:50:07.024907 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:50:07.025012 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:50:07.025419 (+ 407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:50:07.026538 (+ 1119us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:50:07.027266 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:50:07.027612 (+ 346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:50:07.027682 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:50:07.030175 (+ 2493us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:50:07.030189 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:50:07.030189 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 13:50:07.030197 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:50:07.031766 (+ 1569us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:50:07.032088 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:50:07.032092 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:50:07.032092 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:50:07.032142 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:50:07.032298 (+ 156us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:50:07.032299 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:50:07.032299 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:50:07.032456 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:50:07.032527 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0924 13:50:07.032642 (+ 115us) server_negotiation.cc:299] Negotiation successful
0924 13:50:07.032731 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":150,"thread_start_us":100,"threads_started":1}
I20260924 13:50:07.032876 27463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:50:07.024578 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.253:43875 (local address 127.0.0.1:59580)
0924 13:50:07.024622 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0924 13:50:07.024633 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0924 13:50:07.024676 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:50:07.025019 (+ 343us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:50:07.025021 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:50:07.025144 (+ 123us) client_negotiation.cc:189] Negotiated authn=SASL
0924 13:50:07.025339 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:50:07.025342 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:50:07.026640 (+ 1298us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:50:07.026641 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:50:07.027193 (+ 552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:50:07.027195 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:50:07.027677 (+ 482us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:50:07.027678 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:50:07.027743 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:50:07.028148 (+ 405us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 13:50:07.028153 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 13:50:07.030078 (+ 1925us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 13:50:07.031861 (+ 1783us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 13:50:07.031862 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 13:50:07.031862 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 13:50:07.032018 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 13:50:07.032194 (+ 176us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 13:50:07.032195 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 13:50:07.032195 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 13:50:07.032246 (+ 51us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 13:50:07.032514 (+ 268us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 13:50:07.032515 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 13:50:07.032552 (+ 37us) client_negotiation.cc:769] Sending connection context
0924 13:50:07.032612 (+ 60us) client_negotiation.cc:240] Negotiation successful
0924 13:50:07.032798 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":27,"mutex_wait_us":80}
I20260924 13:50:07.033506 27461 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.149.252" port: 33119 } has no permanent_uuid. Determining permanent_uuid...
I20260924 13:50:07.033949 27463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:50:07.033807 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.252:33119 (local address 127.0.0.1:59202)
0924 13:50:07.033860 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0924 13:50:07.033919 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.149.252:33119: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260924 13:50:07.034106 27410 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.13.149.252:33119: connect: Connection refused (error 111)
I20260924 13:50:07.034623 27463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:50:07.034539 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.252:33119 (local address 127.0.0.1:59216)
0924 13:50:07.034575 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0924 13:50:07.034606 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.149.252:33119: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
W20260924 13:50:07.034725 27461 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.149.252:33119: Network error: Client connection negotiation failed: client connection to 127.13.149.252:33119: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.149.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.252.keytab.
Entry for principal kudu/127.13.149.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.252.keytab.
I20260924 13:50:07.044155 13911 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJN9WRF/build/release/bin/kudu
/tmp/dist-test-taskJN9WRF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790257629041645-13911-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790257629041645-13911-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790257629041645-13911-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790257629041645-13911-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.149.252:33119
--webserver_interface=127.13.149.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.252.keytab
--principal=kudu/127.13.149.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.149.212:45357
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.149.254:46275,127.13.149.253:43875,127.13.149.252:33119
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790257629041645-13911-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790257629041645-13911-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790257629041645-13911-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260924 13:50:07.083985 27461 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.149.252" port: 33119 } attempt: 1
I20260924 13:50:07.084595 27463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:50:07.084435 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.252:33119 (local address 127.0.0.1:59218)
0924 13:50:07.084499 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0924 13:50:07.084539 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.149.252:33119: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260924 13:50:07.084982 27463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:50:07.084865 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.252:33119 (local address 127.0.0.1:59220)
0924 13:50:07.084905 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0924 13:50:07.084953 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.149.252:33119: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
W20260924 13:50:07.085083 27461 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.149.252:33119: Network error: Client connection negotiation failed: client connection to 127.13.149.252:33119: connect: Connection refused (error 111)
I20260924 13:50:07.089058 27386 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.149.253" port: 43875 } attempt: 3
Sep 24 13:50:07 dist-test-slave-sjl2 krb5kdc[27305](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257806, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.149.254@KRBTEST.COM for kudu/127.13.149.253@KRBTEST.COM
I20260924 13:50:07.096124 27388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:50:07.089435 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.253:43875 (local address 127.0.0.1:59586)
0924 13:50:07.089502 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0924 13:50:07.089513 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0924 13:50:07.089561 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:50:07.089685 (+ 124us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:50:07.089686 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:50:07.089852 (+ 166us) client_negotiation.cc:189] Negotiated authn=SASL
0924 13:50:07.089921 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:50:07.089922 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:50:07.090586 (+ 664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:50:07.090587 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:50:07.091161 (+ 574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:50:07.091163 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:50:07.091540 (+ 377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:50:07.091541 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:50:07.091632 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:50:07.092066 (+ 434us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 13:50:07.092071 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 13:50:07.093644 (+ 1573us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 13:50:07.095177 (+ 1533us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 13:50:07.095178 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 13:50:07.095179 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 13:50:07.095344 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 13:50:07.095509 (+ 165us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 13:50:07.095510 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 13:50:07.095510 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 13:50:07.095565 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 13:50:07.095796 (+ 231us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 13:50:07.095797 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 13:50:07.095847 (+ 50us) client_negotiation.cc:769] Sending connection context
0924 13:50:07.095904 (+ 57us) client_negotiation.cc:240] Negotiation successful
0924 13:50:07.096016 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":45}
I20260924 13:50:07.096124 27465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:50:07.089516 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59586 (local address 127.13.149.253:43875)
0924 13:50:07.089584 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0924 13:50:07.089586 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:50:07.089597 (+ 11us) server_negotiation.cc:407] Connection header received
0924 13:50:07.089623 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:50:07.089624 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:50:07.089634 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:50:07.089681 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:50:07.089994 (+ 313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:50:07.090505 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:50:07.091263 (+ 758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:50:07.091474 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:50:07.091532 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:50:07.093762 (+ 2230us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:50:07.093764 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:50:07.093764 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 13:50:07.093773 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:50:07.095092 (+ 1319us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:50:07.095404 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:50:07.095405 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:50:07.095406 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:50:07.095456 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:50:07.095613 (+ 157us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:50:07.095614 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:50:07.095614 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:50:07.095733 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:50:07.095789 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0924 13:50:07.095931 (+ 142us) server_negotiation.cc:299] Negotiation successful
0924 13:50:07.096030 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260924 13:50:07.096481 27386 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.149.252" port: 33119 } has no permanent_uuid. Determining permanent_uuid...
I20260924 13:50:07.096908 27388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:50:07.096802 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.252:33119 (local address 127.0.0.1:59234)
0924 13:50:07.096858 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0924 13:50:07.096888 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.149.252:33119: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260924 13:50:07.097234 27388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:50:07.097152 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.252:33119 (local address 127.0.0.1:59250)
0924 13:50:07.097187 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0924 13:50:07.097212 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.149.252:33119: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
W20260924 13:50:07.097316 27386 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.149.252:33119: Network error: Client connection negotiation failed: client connection to 127.13.149.252:33119: connect: Connection refused (error 111)
I20260924 13:50:07.128463 27386 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.149.252" port: 33119 } attempt: 1
I20260924 13:50:07.128980 27388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:50:07.128828 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.252:33119 (local address 127.0.0.1:59262)
0924 13:50:07.128889 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0924 13:50:07.128944 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.149.252:33119: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260924 13:50:07.129288 27388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:50:07.129205 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.252:33119 (local address 127.0.0.1:59276)
0924 13:50:07.129243 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0924 13:50:07.129267 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.149.252:33119: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
W20260924 13:50:07.129391 27386 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.149.252:33119: Network error: Client connection negotiation failed: client connection to 127.13.149.252:33119: connect: Connection refused (error 111)
W20260924 13:50:07.129891 27468 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 13:50:07.130097 27468 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 13:50:07.130131 27468 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 13:50:07.131389 27468 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 13:50:07.131436 27468 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 13:50:07.131455 27468 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 13:50:07.131471 27468 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 13:50:07.131487 27468 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 13:50:07.133170 27468 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.149.212:45357
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790257629041645-13911-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790257629041645-13911-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.13.149.254:46275,127.13.149.253:43875,127.13.149.252:33119
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.252.keytab
--principal=kudu/127.13.149.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.149.252:33119
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790257629041645-13911-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.13.149.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27468
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790257629041645-13911-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 12:29:37 UTC on ccycloud.quasar-iqzbxn.root.comops.site
build id 4670106
I20260924 13:50:07.133438 27468 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 13:50:07.133663 27468 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 13:50:07.138602 27474 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 13:50:07.138595 27473 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 13:50:07.138625 27476 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 13:50:07.138875 27468 server_base.cc:1034] running on GCE node
I20260924 13:50:07.139176 27468 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 13:50:07.139451 27468 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 13:50:07.140605 27468 hybrid_clock.cc:648] HybridClock initialized: now 1790257807140589 us; error 27 us; skew 500 ppm
Sep 24 13:50:07 dist-test-slave-sjl2 krb5kdc[27305](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257807, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.149.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 13:50:07.143167 27468 init.cc:377] Logged in from keytab as kudu/127.13.149.252@KRBTEST.COM (short username kudu)
I20260924 13:50:07.143601 27468 webserver.cc:460] Webserver started at http://127.13.149.252:41751/ using document root <none> and password file <none>
I20260924 13:50:07.143807 27468 fs_manager.cc:359] Metadata directory not provided
I20260924 13:50:07.143847 27468 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 13:50:07.143934 27468 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 13:50:07.144992 27468 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790257629041645-13911-0/minicluster-data/master-2/data/instance:
uuid: "d220a704f0ee4359927bb5f26364016e"
format_stamp: "Formatted at 2026-09-24 13:50:07 on dist-test-slave-sjl2"
server_key: "e56196781648a950211032fdcfb1b22c"
server_key_iv: "3aab975f98d242f1a40b7288ee664555"
server_key_version: "encryptionkey@0"
I20260924 13:50:07.145304 27468 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790257629041645-13911-0/minicluster-data/master-2/wal/instance:
uuid: "d220a704f0ee4359927bb5f26364016e"
format_stamp: "Formatted at 2026-09-24 13:50:07 on dist-test-slave-sjl2"
server_key: "e56196781648a950211032fdcfb1b22c"
server_key_iv: "3aab975f98d242f1a40b7288ee664555"
server_key_version: "encryptionkey@0"
I20260924 13:50:07.146576 27468 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260924 13:50:07.147387 27483 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 13:50:07.147655 27468 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260924 13:50:07.147743 27468 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790257629041645-13911-0/minicluster-data/master-2/data,/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790257629041645-13911-0/minicluster-data/master-2/wal
uuid: "d220a704f0ee4359927bb5f26364016e"
format_stamp: "Formatted at 2026-09-24 13:50:07 on dist-test-slave-sjl2"
server_key: "e56196781648a950211032fdcfb1b22c"
server_key_iv: "3aab975f98d242f1a40b7288ee664555"
server_key_version: "encryptionkey@0"
I20260924 13:50:07.147804 27468 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790257629041645-13911-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790257629041645-13911-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790257629041645-13911-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 13:50:07.153213 27461 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.149.252" port: 33119 } attempt: 2
I20260924 13:50:07.153607 27463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:50:07.153500 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.252:33119 (local address 127.0.0.1:59284)
0924 13:50:07.153550 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0924 13:50:07.153581 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.149.252:33119: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
I20260924 13:50:07.153891 27463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:50:07.153818 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.252:33119 (local address 127.0.0.1:59288)
0924 13:50:07.153851 (+ 33us) negotiation.cc:107] Waiting for socket to connect
0924 13:50:07.153874 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.149.252:33119: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":21}
W20260924 13:50:07.153966 27461 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.149.252:33119: Network error: Client connection negotiation failed: client connection to 127.13.149.252:33119: connect: Connection refused (error 111)
I20260924 13:50:07.161712 27468 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 13:50:07.164040 27468 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 13:50:07.164192 27468 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 13:50:07.168131 27468 rpc_server.cc:307] RPC server started. Bound to: 127.13.149.252:33119
I20260924 13:50:07.168170 27535 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.149.252:33119 every 8 connection(s)
I20260924 13:50:07.168501 27468 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790257629041645-13911-0/minicluster-data/master-2/data/info.pb
I20260924 13:50:07.169392 13911 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJN9WRF/build/release/bin/kudu as pid 27468
I20260924 13:50:07.169420 27536 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 13:50:07.169492 13911 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790257629041645-13911-0/minicluster-data/master-2/wal/instance
I20260924 13:50:07.169699 13911 external_mini_cluster.cc:1428] Setting key cf4bbc523c62837a0b3a18d7e59b9806
I20260924 13:50:07.171854 27536 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.149.254" port: 46275 } has no permanent_uuid. Determining permanent_uuid...
Sep 24 13:50:07 dist-test-slave-sjl2 krb5kdc[27305](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257806, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.149.252@KRBTEST.COM
Sep 24 13:50:07 dist-test-slave-sjl2 krb5kdc[27305](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257807, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.149.252@KRBTEST.COM for kudu/127.13.149.254@KRBTEST.COM
I20260924 13:50:07.180543 27389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:50:07.172620 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46428 (local address 127.13.149.254:46275)
0924 13:50:07.172683 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0924 13:50:07.172684 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0924 13:50:07.172844 (+ 160us) server_negotiation.cc:407] Connection header received
0924 13:50:07.172900 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:50:07.172901 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:50:07.172911 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:50:07.172992 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:50:07.173578 (+ 586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:50:07.174134 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:50:07.175155 (+ 1021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:50:07.175439 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:50:07.175511 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:50:07.177422 (+ 1911us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:50:07.177423 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:50:07.177424 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 13:50:07.177431 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:50:07.179092 (+ 1661us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:50:07.179763 (+ 671us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:50:07.179764 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:50:07.179765 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:50:07.179822 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:50:07.180011 (+ 189us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:50:07.180012 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:50:07.180012 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:50:07.180142 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:50:07.180203 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0924 13:50:07.180408 (+ 205us) server_negotiation.cc:299] Negotiation successful
0924 13:50:07.180478 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260924 13:50:07.180651 27540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:50:07.172541 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:46275 (local address 127.0.0.1:46428)
0924 13:50:07.172762 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0924 13:50:07.172790 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0924 13:50:07.172851 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:50:07.173017 (+ 166us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:50:07.173020 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:50:07.173413 (+ 393us) client_negotiation.cc:189] Negotiated authn=SASL
0924 13:50:07.173496 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:50:07.173498 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:50:07.174231 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:50:07.174232 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:50:07.175077 (+ 845us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:50:07.175080 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:50:07.175517 (+ 437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:50:07.175518 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:50:07.175572 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:50:07.175981 (+ 409us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 13:50:07.175989 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0924 13:50:07.177336 (+ 1347us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 13:50:07.179303 (+ 1967us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 13:50:07.179321 (+ 18us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 13:50:07.179325 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0924 13:50:07.179689 (+ 364us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 13:50:07.179887 (+ 198us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 13:50:07.179888 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 13:50:07.179888 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 13:50:07.179952 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 13:50:07.180231 (+ 279us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 13:50:07.180237 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 13:50:07.180283 (+ 46us) client_negotiation.cc:769] Sending connection context
0924 13:50:07.180372 (+ 89us) client_negotiation.cc:240] Negotiation successful
0924 13:50:07.180472 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":200,"thread_start_us":99,"threads_started":1}
I20260924 13:50:07.180651 27539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:50:07.170208 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59304 (local address 127.13.149.252:33119)
0924 13:50:07.170383 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0924 13:50:07.170393 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0924 13:50:07.170427 (+ 34us) server_negotiation.cc:407] Connection header received
0924 13:50:07.170898 (+ 471us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:50:07.170908 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:50:07.170972 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:50:07.171126 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:50:07.171712 (+ 586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:50:07.172983 (+ 1271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:50:07.173874 (+ 891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:50:07.174505 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:50:07.174603 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:50:07.176969 (+ 2366us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:50:07.176978 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:50:07.176978 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 13:50:07.176985 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:50:07.178667 (+ 1682us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:50:07.179029 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:50:07.179033 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:50:07.179033 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:50:07.179134 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:50:07.179563 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:50:07.179564 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:50:07.179564 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:50:07.179848 (+ 284us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:50:07.179969 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0924 13:50:07.180103 (+ 134us) server_negotiation.cc:299] Negotiation successful
0924 13:50:07.180396 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":122,"thread_start_us":76,"threads_started":1}
I20260924 13:50:07.181270 27536 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.149.253" port: 43875 } has no permanent_uuid. Determining permanent_uuid...
Sep 24 13:50:07 dist-test-slave-sjl2 krb5kdc[27305](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257807, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.149.252@KRBTEST.COM for kudu/127.13.149.253@KRBTEST.COM
Sep 24 13:50:07 dist-test-slave-sjl2 krb5kdc[27305](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257806, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.149.254@KRBTEST.COM
I20260924 13:50:07.188107 27465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:50:07.181540 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59596 (local address 127.13.149.253:43875)
0924 13:50:07.181578 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0924 13:50:07.181580 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:50:07.181589 (+ 9us) server_negotiation.cc:407] Connection header received
0924 13:50:07.181614 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:50:07.181615 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:50:07.181624 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:50:07.181668 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:50:07.181909 (+ 241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:50:07.182380 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:50:07.183018 (+ 638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:50:07.183225 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:50:07.183286 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:50:07.185220 (+ 1934us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:50:07.185222 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:50:07.185222 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 13:50:07.185231 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:50:07.186988 (+ 1757us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:50:07.187341 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:50:07.187343 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:50:07.187343 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:50:07.187405 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:50:07.187560 (+ 155us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:50:07.187561 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:50:07.187561 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:50:07.187709 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:50:07.187760 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0924 13:50:07.187964 (+ 204us) server_negotiation.cc:299] Negotiation successful
0924 13:50:07.188036 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260924 13:50:07.188156 27540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:50:07.181488 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.253:43875 (local address 127.0.0.1:59596)
0924 13:50:07.181521 (+ 33us) negotiation.cc:107] Waiting for socket to connect
0924 13:50:07.181531 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 13:50:07.181569 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:50:07.181669 (+ 100us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:50:07.181670 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:50:07.181806 (+ 136us) client_negotiation.cc:189] Negotiated authn=SASL
0924 13:50:07.181842 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:50:07.181843 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:50:07.182444 (+ 601us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:50:07.182445 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:50:07.182948 (+ 503us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:50:07.182950 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:50:07.183286 (+ 336us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:50:07.183287 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:50:07.183343 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:50:07.183718 (+ 375us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 13:50:07.183722 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0924 13:50:07.185129 (+ 1407us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 13:50:07.187076 (+ 1947us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 13:50:07.187078 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 13:50:07.187078 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 13:50:07.187274 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 13:50:07.187460 (+ 186us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 13:50:07.187461 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 13:50:07.187461 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 13:50:07.187507 (+ 46us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 13:50:07.187793 (+ 286us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 13:50:07.187794 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 13:50:07.187853 (+ 59us) client_negotiation.cc:769] Sending connection context
0924 13:50:07.187932 (+ 79us) client_negotiation.cc:240] Negotiation successful
0924 13:50:07.188053 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17}
I20260924 13:50:07.188699 27389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:50:07.181433 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46436 (local address 127.13.149.254:46275)
0924 13:50:07.181474 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0924 13:50:07.181476 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:50:07.181527 (+ 51us) server_negotiation.cc:407] Connection header received
0924 13:50:07.181608 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:50:07.181609 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:50:07.181619 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:50:07.181662 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:50:07.182100 (+ 438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:50:07.182548 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:50:07.183180 (+ 632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:50:07.183390 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:50:07.183441 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:50:07.185920 (+ 2479us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:50:07.185921 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:50:07.185922 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 13:50:07.185931 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:50:07.187296 (+ 1365us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:50:07.187691 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:50:07.187692 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:50:07.187692 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:50:07.187760 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:50:07.187951 (+ 191us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:50:07.187951 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:50:07.187952 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:50:07.188146 (+ 194us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:50:07.188334 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0924 13:50:07.188522 (+ 188us) server_negotiation.cc:299] Negotiation successful
0924 13:50:07.188602 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260924 13:50:07.188702 27536 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.149.252" port: 33119 } has no permanent_uuid. Determining permanent_uuid...
Sep 24 13:50:07 dist-test-slave-sjl2 krb5kdc[27305](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257807, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.149.252@KRBTEST.COM for kudu/127.13.149.252@KRBTEST.COM
Sep 24 13:50:07 dist-test-slave-sjl2 krb5kdc[27305](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257806, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.149.253@KRBTEST.COM
I20260924 13:50:07.195693 27539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:50:07.189015 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59314 (local address 127.13.149.252:33119)
0924 13:50:07.189053 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0924 13:50:07.189055 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:50:07.189098 (+ 43us) server_negotiation.cc:407] Connection header received
0924 13:50:07.189181 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:50:07.189182 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:50:07.189192 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:50:07.189244 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:50:07.189529 (+ 285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:50:07.190074 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:50:07.190845 (+ 771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:50:07.191050 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:50:07.191104 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:50:07.193111 (+ 2007us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:50:07.193112 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:50:07.193113 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 13:50:07.193119 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:50:07.194595 (+ 1476us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:50:07.194959 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:50:07.194960 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:50:07.194961 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:50:07.195018 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:50:07.195180 (+ 162us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:50:07.195181 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:50:07.195182 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:50:07.195318 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:50:07.195379 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0924 13:50:07.195524 (+ 145us) server_negotiation.cc:299] Negotiation successful
0924 13:50:07.195596 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260924 13:50:07.195808 27540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:50:07.188978 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.252:33119 (local address 127.0.0.1:59314)
0924 13:50:07.189056 (+ 78us) negotiation.cc:107] Waiting for socket to connect
0924 13:50:07.189067 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0924 13:50:07.189114 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:50:07.189259 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:50:07.189260 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:50:07.189411 (+ 151us) client_negotiation.cc:189] Negotiated authn=SASL
0924 13:50:07.189458 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:50:07.189460 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:50:07.190149 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:50:07.190149 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:50:07.190749 (+ 600us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:50:07.190751 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:50:07.191117 (+ 366us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:50:07.191118 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:50:07.191168 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:50:07.191628 (+ 460us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 13:50:07.191632 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0924 13:50:07.193040 (+ 1408us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 13:50:07.194693 (+ 1653us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 13:50:07.194694 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 13:50:07.194695 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 13:50:07.194889 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 13:50:07.195079 (+ 190us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 13:50:07.195079 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 13:50:07.195080 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 13:50:07.195125 (+ 45us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 13:50:07.195382 (+ 257us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 13:50:07.195383 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 13:50:07.195430 (+ 47us) client_negotiation.cc:769] Sending connection context
0924 13:50:07.195503 (+ 73us) client_negotiation.cc:240] Negotiation successful
0924 13:50:07.195690 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":62,"mutex_wait_us":62}
I20260924 13:50:07.196692 27465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:50:07.189310 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59610 (local address 127.13.149.253:43875)
0924 13:50:07.189344 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0924 13:50:07.189346 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:50:07.189353 (+ 7us) server_negotiation.cc:407] Connection header received
0924 13:50:07.189428 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:50:07.189429 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:50:07.189435 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:50:07.189471 (+ 36us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:50:07.189925 (+ 454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:50:07.190428 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:50:07.191233 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:50:07.191404 (+ 171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:50:07.191452 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:50:07.193859 (+ 2407us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:50:07.193860 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:50:07.193861 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 13:50:07.193867 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:50:07.195302 (+ 1435us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:50:07.195632 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:50:07.195633 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:50:07.195634 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:50:07.195714 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:50:07.196088 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:50:07.196089 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:50:07.196089 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:50:07.196218 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:50:07.196404 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0924 13:50:07.196443 (+ 39us) server_negotiation.cc:299] Negotiation successful
0924 13:50:07.196512 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260924 13:50:07.197549 27536 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d220a704f0ee4359927bb5f26364016e: Bootstrap starting.
I20260924 13:50:07.198304 13911 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260924 13:50:07.198544 27536 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d220a704f0ee4359927bb5f26364016e: Neither blocks nor log segments found. Creating new log.
I20260924 13:50:07.198846 27536 log.cc:824] T 00000000000000000000000000000000 P d220a704f0ee4359927bb5f26364016e: Log is configured to *not* fsync() on all Append() calls
I20260924 13:50:07.199971 27536 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d220a704f0ee4359927bb5f26364016e: No bootstrap required, opened a new log
I20260924 13:50:07.202142 27536 raft_consensus.cc:348] T 00000000000000000000000000000000 P d220a704f0ee4359927bb5f26364016e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eef9f241ec1746809efe5a82656b136f" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 46275 } } peers { permanent_uuid: "25e330f2c1e8482f9eb2a0cb3837e54a" member_type: VOTER last_known_addr { host: "127.13.149.253" port: 43875 } } peers { permanent_uuid: "d220a704f0ee4359927bb5f26364016e" member_type: VOTER last_known_addr { host: "127.13.149.252" port: 33119 } }
I20260924 13:50:07.202324 27536 raft_consensus.cc:374] T 00000000000000000000000000000000 P d220a704f0ee4359927bb5f26364016e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 13:50:07.202358 27536 raft_consensus.cc:729] T 00000000000000000000000000000000 P d220a704f0ee4359927bb5f26364016e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d220a704f0ee4359927bb5f26364016e, State: Initialized, Role: FOLLOWER
I20260924 13:50:07.202476 27536 consensus_queue.cc:260] T 00000000000000000000000000000000 P d220a704f0ee4359927bb5f26364016e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eef9f241ec1746809efe5a82656b136f" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 46275 } } peers { permanent_uuid: "25e330f2c1e8482f9eb2a0cb3837e54a" member_type: VOTER last_known_addr { host: "127.13.149.253" port: 43875 } } peers { permanent_uuid: "d220a704f0ee4359927bb5f26364016e" member_type: VOTER last_known_addr { host: "127.13.149.252" port: 33119 } }
I20260924 13:50:07.202792 27536 sys_catalog.cc:564] T 00000000000000000000000000000000 P d220a704f0ee4359927bb5f26364016e [sys.catalog]: configured and running, proceeding with master startup.
I20260924 13:50:07.203660 27549 sys_catalog.cc:455] T 00000000000000000000000000000000 P d220a704f0ee4359927bb5f26364016e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eef9f241ec1746809efe5a82656b136f" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 46275 } } peers { permanent_uuid: "25e330f2c1e8482f9eb2a0cb3837e54a" member_type: VOTER last_known_addr { host: "127.13.149.253" port: 43875 } } peers { permanent_uuid: "d220a704f0ee4359927bb5f26364016e" member_type: VOTER last_known_addr { host: "127.13.149.252" port: 33119 } } }
I20260924 13:50:07.204521 27386 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.149.252" port: 33119 } attempt: 2
I20260924 13:50:07.208325 27465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:50:07.199998 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59612 (local address 127.13.149.253:43875)
0924 13:50:07.200036 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0924 13:50:07.200038 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:50:07.200356 (+ 318us) server_negotiation.cc:407] Connection header received
0924 13:50:07.200429 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:50:07.200430 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:50:07.200439 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:50:07.200464 (+ 25us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:50:07.201227 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:50:07.201917 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:50:07.202904 (+ 987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:50:07.203176 (+ 272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:50:07.203218 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:50:07.204796 (+ 1578us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:50:07.204797 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:50:07.204797 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 13:50:07.204806 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:50:07.207077 (+ 2271us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:50:07.207489 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:50:07.207491 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:50:07.207491 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:50:07.207548 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:50:07.207730 (+ 182us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:50:07.207731 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:50:07.207731 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:50:07.207861 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:50:07.207930 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0924 13:50:07.208135 (+ 205us) server_negotiation.cc:299] Negotiation successful
0924 13:50:07.208230 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260924 13:50:07.205570 27468 master_runner.cc:186] Error getting master registration for 127.13.149.254:46275: OK, instance_id { permanent_uuid: "eef9f241ec1746809efe5a82656b136f" instance_seqno: 1790257806862023 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260924 13:50:07.208523 27468 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260924 13:50:07.208578 27549 sys_catalog.cc:458] T 00000000000000000000000000000000 P d220a704f0ee4359927bb5f26364016e [sys.catalog]: This master's current role is: FOLLOWER
I20260924 13:50:07.209048 27539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:50:07.199760 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59324 (local address 127.13.149.252:33119)
0924 13:50:07.199798 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0924 13:50:07.199801 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 13:50:07.200054 (+ 253us) server_negotiation.cc:407] Connection header received
0924 13:50:07.200143 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:50:07.200144 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:50:07.200153 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:50:07.200180 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:50:07.201065 (+ 885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:50:07.201827 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:50:07.203210 (+ 1383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:50:07.203472 (+ 262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:50:07.203528 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:50:07.205263 (+ 1735us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:50:07.205264 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:50:07.205264 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 13:50:07.205272 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:50:07.206940 (+ 1668us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:50:07.208018 (+ 1078us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:50:07.208019 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:50:07.208019 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:50:07.208074 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:50:07.208250 (+ 176us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:50:07.208251 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:50:07.208251 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:50:07.208403 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:50:07.208483 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0924 13:50:07.208855 (+ 372us) server_negotiation.cc:299] Negotiation successful
0924 13:50:07.208965 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260924 13:50:07.209445 27389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:50:07.199787 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46442 (local address 127.13.149.254:46275)
0924 13:50:07.199831 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0924 13:50:07.199832 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0924 13:50:07.199960 (+ 128us) server_negotiation.cc:407] Connection header received
0924 13:50:07.200138 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:50:07.200139 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:50:07.200151 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:50:07.200182 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:50:07.201078 (+ 896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:50:07.201814 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:50:07.202992 (+ 1178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:50:07.203299 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:50:07.203339 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:50:07.205548 (+ 2209us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:50:07.205550 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:50:07.205550 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 13:50:07.205558 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:50:07.207594 (+ 2036us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:50:07.208102 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:50:07.208104 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:50:07.208104 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:50:07.208156 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:50:07.208339 (+ 183us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:50:07.208340 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:50:07.208340 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:50:07.208455 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:50:07.208526 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0924 13:50:07.209288 (+ 762us) server_negotiation.cc:299] Negotiation successful
0924 13:50:07.209366 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
W20260924 13:50:07.209478 27560 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d220a704f0ee4359927bb5f26364016e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260924 13:50:07.209519 27560 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
Sep 24 13:50:07 dist-test-slave-sjl2 krb5kdc[27305](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257806, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.149.254@KRBTEST.COM for kudu/127.13.149.252@KRBTEST.COM
I20260924 13:50:07.214026 27388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:50:07.204762 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.252:33119 (local address 127.0.0.1:59338)
0924 13:50:07.204854 (+ 92us) negotiation.cc:107] Waiting for socket to connect
0924 13:50:07.204865 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0924 13:50:07.204901 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:50:07.205733 (+ 832us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:50:07.205734 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:50:07.205871 (+ 137us) client_negotiation.cc:189] Negotiated authn=SASL
0924 13:50:07.205921 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:50:07.205923 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:50:07.206942 (+ 1019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:50:07.206943 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:50:07.207739 (+ 796us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:50:07.207742 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:50:07.209943 (+ 2201us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:50:07.209944 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:50:07.210003 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:50:07.210422 (+ 419us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 13:50:07.210427 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 13:50:07.211515 (+ 1088us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 13:50:07.213100 (+ 1585us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 13:50:07.213101 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 13:50:07.213101 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 13:50:07.213256 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 13:50:07.213434 (+ 178us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 13:50:07.213435 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 13:50:07.213435 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 13:50:07.213490 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 13:50:07.213710 (+ 220us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 13:50:07.213710 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 13:50:07.213763 (+ 53us) client_negotiation.cc:769] Sending connection context
0924 13:50:07.213821 (+ 58us) client_negotiation.cc:240] Negotiation successful
0924 13:50:07.213921 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":74}
I20260924 13:50:07.214039 27561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:50:07.205004 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59338 (local address 127.13.149.252:33119)
0924 13:50:07.205598 (+ 594us) server_negotiation.cc:206] Beginning negotiation
0924 13:50:07.205600 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:50:07.205606 (+ 6us) server_negotiation.cc:407] Connection header received
0924 13:50:07.205623 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:50:07.205623 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:50:07.205642 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:50:07.205988 (+ 346us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:50:07.206047 (+ 59us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:50:07.206842 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:50:07.209659 (+ 2817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:50:07.209868 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:50:07.209927 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:50:07.211611 (+ 1684us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:50:07.211612 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:50:07.211613 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 13:50:07.211619 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:50:07.213023 (+ 1404us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:50:07.213312 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:50:07.213313 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:50:07.213314 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:50:07.213380 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:50:07.213540 (+ 160us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:50:07.213540 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:50:07.213541 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:50:07.213653 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:50:07.213708 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0924 13:50:07.213863 (+ 155us) server_negotiation.cc:299] Negotiation successful
0924 13:50:07.213935 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":572,"thread_start_us":74,"threads_started":1}
I20260924 13:50:07.215770 27386 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eef9f241ec1746809efe5a82656b136f: Bootstrap starting.
I20260924 13:50:07.216578 27386 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P eef9f241ec1746809efe5a82656b136f: Neither blocks nor log segments found. Creating new log.
I20260924 13:50:07.216902 27386 log.cc:824] T 00000000000000000000000000000000 P eef9f241ec1746809efe5a82656b136f: Log is configured to *not* fsync() on all Append() calls
I20260924 13:50:07.217850 27386 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eef9f241ec1746809efe5a82656b136f: No bootstrap required, opened a new log
I20260924 13:50:07.219902 27386 raft_consensus.cc:348] T 00000000000000000000000000000000 P eef9f241ec1746809efe5a82656b136f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eef9f241ec1746809efe5a82656b136f" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 46275 } } peers { permanent_uuid: "25e330f2c1e8482f9eb2a0cb3837e54a" member_type: VOTER last_known_addr { host: "127.13.149.253" port: 43875 } } peers { permanent_uuid: "d220a704f0ee4359927bb5f26364016e" member_type: VOTER last_known_addr { host: "127.13.149.252" port: 33119 } }
I20260924 13:50:07.220050 27386 raft_consensus.cc:374] T 00000000000000000000000000000000 P eef9f241ec1746809efe5a82656b136f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 13:50:07.220113 27386 raft_consensus.cc:729] T 00000000000000000000000000000000 P eef9f241ec1746809efe5a82656b136f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eef9f241ec1746809efe5a82656b136f, State: Initialized, Role: FOLLOWER
I20260924 13:50:07.220242 27386 consensus_queue.cc:260] T 00000000000000000000000000000000 P eef9f241ec1746809efe5a82656b136f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eef9f241ec1746809efe5a82656b136f" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 46275 } } peers { permanent_uuid: "25e330f2c1e8482f9eb2a0cb3837e54a" member_type: VOTER last_known_addr { host: "127.13.149.253" port: 43875 } } peers { permanent_uuid: "d220a704f0ee4359927bb5f26364016e" member_type: VOTER last_known_addr { host: "127.13.149.252" port: 33119 } }
I20260924 13:50:07.220585 27386 sys_catalog.cc:564] T 00000000000000000000000000000000 P eef9f241ec1746809efe5a82656b136f [sys.catalog]: configured and running, proceeding with master startup.
I20260924 13:50:07.220546 27564 sys_catalog.cc:455] T 00000000000000000000000000000000 P eef9f241ec1746809efe5a82656b136f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eef9f241ec1746809efe5a82656b136f" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 46275 } } peers { permanent_uuid: "25e330f2c1e8482f9eb2a0cb3837e54a" member_type: VOTER last_known_addr { host: "127.13.149.253" port: 43875 } } peers { permanent_uuid: "d220a704f0ee4359927bb5f26364016e" member_type: VOTER last_known_addr { host: "127.13.149.252" port: 33119 } } }
I20260924 13:50:07.220698 27564 sys_catalog.cc:458] T 00000000000000000000000000000000 P eef9f241ec1746809efe5a82656b136f [sys.catalog]: This master's current role is: FOLLOWER
W20260924 13:50:07.222493 27575 catalog_manager.cc:1558] T 00000000000000000000000000000000 P eef9f241ec1746809efe5a82656b136f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260924 13:50:07.222551 27575 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260924 13:50:07.223268 27318 master_runner.cc:186] Error getting master registration for 127.13.149.253:43875: OK, instance_id { permanent_uuid: "25e330f2c1e8482f9eb2a0cb3837e54a" instance_seqno: 1790257807009493 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260924 13:50:07.225622 27318 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260924 13:50:07.245129 27461 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.149.252" port: 33119 } attempt: 3
Sep 24 13:50:07 dist-test-slave-sjl2 krb5kdc[27305](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257806, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.149.253@KRBTEST.COM for kudu/127.13.149.252@KRBTEST.COM
I20260924 13:50:07.252051 27463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:50:07.245488 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.252:33119 (local address 127.0.0.1:59352)
0924 13:50:07.245547 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0924 13:50:07.245559 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0924 13:50:07.245623 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:50:07.245757 (+ 134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:50:07.245758 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:50:07.245923 (+ 165us) client_negotiation.cc:189] Negotiated authn=SASL
0924 13:50:07.245992 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:50:07.245994 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:50:07.246799 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:50:07.246801 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:50:07.247455 (+ 654us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:50:07.247458 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:50:07.247818 (+ 360us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:50:07.247820 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:50:07.247874 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:50:07.248296 (+ 422us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 13:50:07.248301 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 13:50:07.249504 (+ 1203us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 13:50:07.251001 (+ 1497us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 13:50:07.251002 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 13:50:07.251003 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 13:50:07.251213 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 13:50:07.251404 (+ 191us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 13:50:07.251405 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 13:50:07.251405 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 13:50:07.251466 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 13:50:07.251735 (+ 269us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 13:50:07.251736 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 13:50:07.251783 (+ 47us) client_negotiation.cc:769] Sending connection context
0924 13:50:07.251839 (+ 56us) client_negotiation.cc:240] Negotiation successful
0924 13:50:07.251948 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38}
I20260924 13:50:07.252051 27561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:50:07.245534 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59352 (local address 127.13.149.252:33119)
0924 13:50:07.245575 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0924 13:50:07.245577 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:50:07.245604 (+ 27us) server_negotiation.cc:407] Connection header received
0924 13:50:07.245686 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:50:07.245687 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:50:07.245697 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:50:07.245742 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:50:07.246065 (+ 323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:50:07.246715 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:50:07.247529 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:50:07.247751 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:50:07.247807 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:50:07.249587 (+ 1780us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:50:07.249588 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:50:07.249589 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 13:50:07.249596 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:50:07.250925 (+ 1329us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:50:07.251278 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:50:07.251279 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:50:07.251279 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:50:07.251344 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:50:07.251524 (+ 180us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:50:07.251525 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:50:07.251526 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:50:07.251652 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:50:07.251717 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0924 13:50:07.251867 (+ 150us) server_negotiation.cc:299] Negotiation successful
0924 13:50:07.251949 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260924 13:50:07.253774 27461 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 25e330f2c1e8482f9eb2a0cb3837e54a: Bootstrap starting.
I20260924 13:50:07.254402 27461 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 25e330f2c1e8482f9eb2a0cb3837e54a: Neither blocks nor log segments found. Creating new log.
I20260924 13:50:07.254715 27461 log.cc:824] T 00000000000000000000000000000000 P 25e330f2c1e8482f9eb2a0cb3837e54a: Log is configured to *not* fsync() on all Append() calls
I20260924 13:50:07.255556 27461 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 25e330f2c1e8482f9eb2a0cb3837e54a: No bootstrap required, opened a new log
I20260924 13:50:07.257061 27461 raft_consensus.cc:348] T 00000000000000000000000000000000 P 25e330f2c1e8482f9eb2a0cb3837e54a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eef9f241ec1746809efe5a82656b136f" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 46275 } } peers { permanent_uuid: "25e330f2c1e8482f9eb2a0cb3837e54a" member_type: VOTER last_known_addr { host: "127.13.149.253" port: 43875 } } peers { permanent_uuid: "d220a704f0ee4359927bb5f26364016e" member_type: VOTER last_known_addr { host: "127.13.149.252" port: 33119 } }
I20260924 13:50:07.257216 27461 raft_consensus.cc:374] T 00000000000000000000000000000000 P 25e330f2c1e8482f9eb2a0cb3837e54a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 13:50:07.257261 27461 raft_consensus.cc:729] T 00000000000000000000000000000000 P 25e330f2c1e8482f9eb2a0cb3837e54a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 25e330f2c1e8482f9eb2a0cb3837e54a, State: Initialized, Role: FOLLOWER
I20260924 13:50:07.257382 27461 consensus_queue.cc:260] T 00000000000000000000000000000000 P 25e330f2c1e8482f9eb2a0cb3837e54a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eef9f241ec1746809efe5a82656b136f" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 46275 } } peers { permanent_uuid: "25e330f2c1e8482f9eb2a0cb3837e54a" member_type: VOTER last_known_addr { host: "127.13.149.253" port: 43875 } } peers { permanent_uuid: "d220a704f0ee4359927bb5f26364016e" member_type: VOTER last_known_addr { host: "127.13.149.252" port: 33119 } }
I20260924 13:50:07.257656 27578 sys_catalog.cc:455] T 00000000000000000000000000000000 P 25e330f2c1e8482f9eb2a0cb3837e54a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eef9f241ec1746809efe5a82656b136f" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 46275 } } peers { permanent_uuid: "25e330f2c1e8482f9eb2a0cb3837e54a" member_type: VOTER last_known_addr { host: "127.13.149.253" port: 43875 } } peers { permanent_uuid: "d220a704f0ee4359927bb5f26364016e" member_type: VOTER last_known_addr { host: "127.13.149.252" port: 33119 } } }
I20260924 13:50:07.257725 27461 sys_catalog.cc:564] T 00000000000000000000000000000000 P 25e330f2c1e8482f9eb2a0cb3837e54a [sys.catalog]: configured and running, proceeding with master startup.
I20260924 13:50:07.257802 27578 sys_catalog.cc:458] T 00000000000000000000000000000000 P 25e330f2c1e8482f9eb2a0cb3837e54a [sys.catalog]: This master's current role is: FOLLOWER
W20260924 13:50:07.259949 27589 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 25e330f2c1e8482f9eb2a0cb3837e54a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260924 13:50:07.260327 27589 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260924 13:50:07.264196 27393 master_runner.cc:263] No leader master found from master 25e330f2c1e8482f9eb2a0cb3837e54a
I20260924 13:50:07.264279 27393 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260924 13:50:07.285820 27564 raft_consensus.cc:482] T 00000000000000000000000000000000 P eef9f241ec1746809efe5a82656b136f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 13:50:07.285991 27564 raft_consensus.cc:504] T 00000000000000000000000000000000 P eef9f241ec1746809efe5a82656b136f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eef9f241ec1746809efe5a82656b136f" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 46275 } } peers { permanent_uuid: "25e330f2c1e8482f9eb2a0cb3837e54a" member_type: VOTER last_known_addr { host: "127.13.149.253" port: 43875 } } peers { permanent_uuid: "d220a704f0ee4359927bb5f26364016e" member_type: VOTER last_known_addr { host: "127.13.149.252" port: 33119 } }
I20260924 13:50:07.286301 27564 leader_election.cc:288] T 00000000000000000000000000000000 P eef9f241ec1746809efe5a82656b136f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 25e330f2c1e8482f9eb2a0cb3837e54a (127.13.149.253:43875), d220a704f0ee4359927bb5f26364016e (127.13.149.252:33119)
I20260924 13:50:07.286943 27435 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "eef9f241ec1746809efe5a82656b136f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25e330f2c1e8482f9eb2a0cb3837e54a" is_pre_election: true
I20260924 13:50:07.287134 27435 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 25e330f2c1e8482f9eb2a0cb3837e54a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eef9f241ec1746809efe5a82656b136f in term 0.
I20260924 13:50:07.287492 27337 leader_election.cc:302] T 00000000000000000000000000000000 P eef9f241ec1746809efe5a82656b136f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 25e330f2c1e8482f9eb2a0cb3837e54a, eef9f241ec1746809efe5a82656b136f; no voters:
I20260924 13:50:07.288004 27510 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "eef9f241ec1746809efe5a82656b136f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d220a704f0ee4359927bb5f26364016e" is_pre_election: true
I20260924 13:50:07.288179 27510 raft_consensus.cc:2454] T 00000000000000000000000000000000 P d220a704f0ee4359927bb5f26364016e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eef9f241ec1746809efe5a82656b136f in term 0.
I20260924 13:50:07.288524 27564 raft_consensus.cc:2781] T 00000000000000000000000000000000 P eef9f241ec1746809efe5a82656b136f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 13:50:07.288609 27564 raft_consensus.cc:482] T 00000000000000000000000000000000 P eef9f241ec1746809efe5a82656b136f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 13:50:07.288636 27564 raft_consensus.cc:3037] T 00000000000000000000000000000000 P eef9f241ec1746809efe5a82656b136f [term 0 FOLLOWER]: Advancing to term 1
I20260924 13:50:07.289806 27564 raft_consensus.cc:504] T 00000000000000000000000000000000 P eef9f241ec1746809efe5a82656b136f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eef9f241ec1746809efe5a82656b136f" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 46275 } } peers { permanent_uuid: "25e330f2c1e8482f9eb2a0cb3837e54a" member_type: VOTER last_known_addr { host: "127.13.149.253" port: 43875 } } peers { permanent_uuid: "d220a704f0ee4359927bb5f26364016e" member_type: VOTER last_known_addr { host: "127.13.149.252" port: 33119 } }
I20260924 13:50:07.289956 27564 leader_election.cc:288] T 00000000000000000000000000000000 P eef9f241ec1746809efe5a82656b136f [CANDIDATE]: Term 1 election: Requested vote from peers 25e330f2c1e8482f9eb2a0cb3837e54a (127.13.149.253:43875), d220a704f0ee4359927bb5f26364016e (127.13.149.252:33119)
I20260924 13:50:07.290383 27435 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "eef9f241ec1746809efe5a82656b136f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25e330f2c1e8482f9eb2a0cb3837e54a"
I20260924 13:50:07.290485 27435 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 25e330f2c1e8482f9eb2a0cb3837e54a [term 0 FOLLOWER]: Advancing to term 1
I20260924 13:50:07.291453 27435 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 25e330f2c1e8482f9eb2a0cb3837e54a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eef9f241ec1746809efe5a82656b136f in term 1.
I20260924 13:50:07.291714 27337 leader_election.cc:302] T 00000000000000000000000000000000 P eef9f241ec1746809efe5a82656b136f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 25e330f2c1e8482f9eb2a0cb3837e54a, eef9f241ec1746809efe5a82656b136f; no voters:
I20260924 13:50:07.291867 27510 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "eef9f241ec1746809efe5a82656b136f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d220a704f0ee4359927bb5f26364016e"
I20260924 13:50:07.291960 27510 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d220a704f0ee4359927bb5f26364016e [term 0 FOLLOWER]: Advancing to term 1
I20260924 13:50:07.292806 27510 raft_consensus.cc:2454] T 00000000000000000000000000000000 P d220a704f0ee4359927bb5f26364016e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eef9f241ec1746809efe5a82656b136f in term 1.
I20260924 13:50:07.293135 27564 raft_consensus.cc:2781] T 00000000000000000000000000000000 P eef9f241ec1746809efe5a82656b136f [term 1 FOLLOWER]: Leader election won for term 1
I20260924 13:50:07.293324 27564 raft_consensus.cc:686] T 00000000000000000000000000000000 P eef9f241ec1746809efe5a82656b136f [term 1 LEADER]: Becoming Leader. State: Replica: eef9f241ec1746809efe5a82656b136f, State: Running, Role: LEADER
I20260924 13:50:07.293426 27564 consensus_queue.cc:237] T 00000000000000000000000000000000 P eef9f241ec1746809efe5a82656b136f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eef9f241ec1746809efe5a82656b136f" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 46275 } } peers { permanent_uuid: "25e330f2c1e8482f9eb2a0cb3837e54a" member_type: VOTER last_known_addr { host: "127.13.149.253" port: 43875 } } peers { permanent_uuid: "d220a704f0ee4359927bb5f26364016e" member_type: VOTER last_known_addr { host: "127.13.149.252" port: 33119 } }
I20260924 13:50:07.293908 27564 sys_catalog.cc:455] T 00000000000000000000000000000000 P eef9f241ec1746809efe5a82656b136f [sys.catalog]: SysCatalogTable state changed. Reason: New leader eef9f241ec1746809efe5a82656b136f. Latest consensus state: current_term: 1 leader_uuid: "eef9f241ec1746809efe5a82656b136f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eef9f241ec1746809efe5a82656b136f" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 46275 } } peers { permanent_uuid: "25e330f2c1e8482f9eb2a0cb3837e54a" member_type: VOTER last_known_addr { host: "127.13.149.253" port: 43875 } } peers { permanent_uuid: "d220a704f0ee4359927bb5f26364016e" member_type: VOTER last_known_addr { host: "127.13.149.252" port: 33119 } } }
I20260924 13:50:07.293978 27564 sys_catalog.cc:458] T 00000000000000000000000000000000 P eef9f241ec1746809efe5a82656b136f [sys.catalog]: This master's current role is: LEADER
I20260924 13:50:07.294553 27593 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 13:50:07.296447 27593 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 13:50:07.298508 27510 raft_consensus.cc:1261] T 00000000000000000000000000000000 P d220a704f0ee4359927bb5f26364016e [term 1 FOLLOWER]: Refusing update from remote peer eef9f241ec1746809efe5a82656b136f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 13:50:07.298954 27564 consensus_queue.cc:1035] T 00000000000000000000000000000000 P eef9f241ec1746809efe5a82656b136f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d220a704f0ee4359927bb5f26364016e" member_type: VOTER last_known_addr { host: "127.13.149.252" port: 33119 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 13:50:07.300418 27549 sys_catalog.cc:455] T 00000000000000000000000000000000 P d220a704f0ee4359927bb5f26364016e [sys.catalog]: SysCatalogTable state changed. Reason: New leader eef9f241ec1746809efe5a82656b136f. Latest consensus state: current_term: 1 leader_uuid: "eef9f241ec1746809efe5a82656b136f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eef9f241ec1746809efe5a82656b136f" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 46275 } } peers { permanent_uuid: "25e330f2c1e8482f9eb2a0cb3837e54a" member_type: VOTER last_known_addr { host: "127.13.149.253" port: 43875 } } peers { permanent_uuid: "d220a704f0ee4359927bb5f26364016e" member_type: VOTER last_known_addr { host: "127.13.149.252" port: 33119 } } }
I20260924 13:50:07.300509 27549 sys_catalog.cc:458] T 00000000000000000000000000000000 P d220a704f0ee4359927bb5f26364016e [sys.catalog]: This master's current role is: FOLLOWER
I20260924 13:50:07.301016 27435 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 25e330f2c1e8482f9eb2a0cb3837e54a [term 1 FOLLOWER]: Refusing update from remote peer eef9f241ec1746809efe5a82656b136f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 13:50:07.301316 27564 consensus_queue.cc:1035] T 00000000000000000000000000000000 P eef9f241ec1746809efe5a82656b136f [LEADER]: Connected to new peer: Peer: permanent_uuid: "25e330f2c1e8482f9eb2a0cb3837e54a" member_type: VOTER last_known_addr { host: "127.13.149.253" port: 43875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 13:50:07.302587 27578 sys_catalog.cc:455] T 00000000000000000000000000000000 P 25e330f2c1e8482f9eb2a0cb3837e54a [sys.catalog]: SysCatalogTable state changed. Reason: New leader eef9f241ec1746809efe5a82656b136f. Latest consensus state: current_term: 1 leader_uuid: "eef9f241ec1746809efe5a82656b136f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eef9f241ec1746809efe5a82656b136f" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 46275 } } peers { permanent_uuid: "25e330f2c1e8482f9eb2a0cb3837e54a" member_type: VOTER last_known_addr { host: "127.13.149.253" port: 43875 } } peers { permanent_uuid: "d220a704f0ee4359927bb5f26364016e" member_type: VOTER last_known_addr { host: "127.13.149.252" port: 33119 } } }
I20260924 13:50:07.302695 27578 sys_catalog.cc:458] T 00000000000000000000000000000000 P 25e330f2c1e8482f9eb2a0cb3837e54a [sys.catalog]: This master's current role is: FOLLOWER
I20260924 13:50:07.304142 27591 sys_catalog.cc:455] T 00000000000000000000000000000000 P eef9f241ec1746809efe5a82656b136f [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "eef9f241ec1746809efe5a82656b136f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eef9f241ec1746809efe5a82656b136f" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 46275 } } peers { permanent_uuid: "25e330f2c1e8482f9eb2a0cb3837e54a" member_type: VOTER last_known_addr { host: "127.13.149.253" port: 43875 } } peers { permanent_uuid: "d220a704f0ee4359927bb5f26364016e" member_type: VOTER last_known_addr { host: "127.13.149.252" port: 33119 } } }
I20260924 13:50:07.304248 27591 sys_catalog.cc:458] T 00000000000000000000000000000000 P eef9f241ec1746809efe5a82656b136f [sys.catalog]: This master's current role is: LEADER
I20260924 13:50:07.304826 27549 sys_catalog.cc:455] T 00000000000000000000000000000000 P d220a704f0ee4359927bb5f26364016e [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "eef9f241ec1746809efe5a82656b136f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eef9f241ec1746809efe5a82656b136f" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 46275 } } peers { permanent_uuid: "25e330f2c1e8482f9eb2a0cb3837e54a" member_type: VOTER last_known_addr { host: "127.13.149.253" port: 43875 } } peers { permanent_uuid: "d220a704f0ee4359927bb5f26364016e" member_type: VOTER last_known_addr { host: "127.13.149.252" port: 33119 } } }
I20260924 13:50:07.304934 27549 sys_catalog.cc:458] T 00000000000000000000000000000000 P d220a704f0ee4359927bb5f26364016e [sys.catalog]: This master's current role is: FOLLOWER
I20260924 13:50:07.305210 27594 mvcc.cc:205] Tried to move back new op lower bound from 7332895978691993600 to 7332895978674536448. Current Snapshot: MvccSnapshot[applied={T|T < 7332895978691993600}]
I20260924 13:50:07.305413 27595 mvcc.cc:205] Tried to move back new op lower bound from 7332895978691993600 to 7332895978674536448. Current Snapshot: MvccSnapshot[applied={T|T < 7332895978691993600}]
I20260924 13:50:07.306313 27593 catalog_manager.cc:1347] Generated new cluster ID: 59dd38b52afd433b8bcff78828c021e3
I20260924 13:50:07.306358 27593 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 13:50:07.314218 27564 sys_catalog.cc:455] T 00000000000000000000000000000000 P eef9f241ec1746809efe5a82656b136f [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "eef9f241ec1746809efe5a82656b136f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eef9f241ec1746809efe5a82656b136f" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 46275 } } peers { permanent_uuid: "25e330f2c1e8482f9eb2a0cb3837e54a" member_type: VOTER last_known_addr { host: "127.13.149.253" port: 43875 } } peers { permanent_uuid: "d220a704f0ee4359927bb5f26364016e" member_type: VOTER last_known_addr { host: "127.13.149.252" port: 33119 } } }
I20260924 13:50:07.314339 27564 sys_catalog.cc:458] T 00000000000000000000000000000000 P eef9f241ec1746809efe5a82656b136f [sys.catalog]: This master's current role is: LEADER
I20260924 13:50:07.314810 27578 sys_catalog.cc:455] T 00000000000000000000000000000000 P 25e330f2c1e8482f9eb2a0cb3837e54a [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "eef9f241ec1746809efe5a82656b136f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eef9f241ec1746809efe5a82656b136f" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 46275 } } peers { permanent_uuid: "25e330f2c1e8482f9eb2a0cb3837e54a" member_type: VOTER last_known_addr { host: "127.13.149.253" port: 43875 } } peers { permanent_uuid: "d220a704f0ee4359927bb5f26364016e" member_type: VOTER last_known_addr { host: "127.13.149.252" port: 33119 } } }
I20260924 13:50:07.314901 27578 sys_catalog.cc:458] T 00000000000000000000000000000000 P 25e330f2c1e8482f9eb2a0cb3837e54a [sys.catalog]: This master's current role is: FOLLOWER
I20260924 13:50:07.315012 27597 mvcc.cc:205] Tried to move back new op lower bound from 7332895978691993600 to 7332895978674536448. Current Snapshot: MvccSnapshot[applied={T|T < 7332895978691993600}]
I20260924 13:50:07.318504 27593 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 13:50:07.319304 27593 catalog_manager.cc:1504] Loading token signing keys...
I20260924 13:50:07.328984 27593 catalog_manager.cc:5868] T 00000000000000000000000000000000 P eef9f241ec1746809efe5a82656b136f: Generated new TSK 0
I20260924 13:50:07.329489 27593 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260924 13:50:08.209956 27560 catalog_manager.cc:1259] Loaded cluster ID: 59dd38b52afd433b8bcff78828c021e3
I20260924 13:50:08.210037 27560 catalog_manager.cc:1552] T 00000000000000000000000000000000 P d220a704f0ee4359927bb5f26364016e: loading cluster ID for follower catalog manager: success
I20260924 13:50:08.210842 27560 catalog_manager.cc:1574] T 00000000000000000000000000000000 P d220a704f0ee4359927bb5f26364016e: acquiring CA information for follower catalog manager: success
I20260924 13:50:08.211125 27560 catalog_manager.cc:1602] T 00000000000000000000000000000000 P d220a704f0ee4359927bb5f26364016e: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260924 13:50:08.260835 27589 catalog_manager.cc:1259] Loaded cluster ID: 59dd38b52afd433b8bcff78828c021e3
I20260924 13:50:08.260934 27589 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 25e330f2c1e8482f9eb2a0cb3837e54a: loading cluster ID for follower catalog manager: success
I20260924 13:50:08.261775 27589 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 25e330f2c1e8482f9eb2a0cb3837e54a: acquiring CA information for follower catalog manager: success
I20260924 13:50:08.262056 27589 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 25e330f2c1e8482f9eb2a0cb3837e54a: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260924 13:50:08.398872 13911 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJN9WRF/build/release/bin/kudu with pid 27318
I20260924 13:50:08.403506 13911 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJN9WRF/build/release/bin/kudu with pid 27393
I20260924 13:50:08.407980 13911 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJN9WRF/build/release/bin/kudu with pid 27468
2026-09-24T13:50:08Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (2746 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790257629041645-13911-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 13:50:08 dist-test-slave-sjl2 krb5kdc[27613](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 13:50:08 dist-test-slave-sjl2 krb5kdc[27613](info): set up 2 sockets
Sep 24 13:50:08 dist-test-slave-sjl2 krb5kdc[27613](info): commencing operation
krb5kdc: starting...
W20260924 13:50:09.365855 13911 mini_kdc.cc:121] Time spent starting KDC: real 0.939s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790257629041645-13911-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790257629041645-13911-0/krb5kdc/test-user.keytab.
Sep 24 13:50:09 dist-test-slave-sjl2 krb5kdc[27613](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257809, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T13:50:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T13:50:09Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.149.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.149.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.149.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.254.keytab.
Entry for principal kudu/127.13.149.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.254.keytab.
I20260924 13:50:09.484063 13911 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJN9WRF/build/release/bin/kudu
/tmp/dist-test-taskJN9WRF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790257629041645-13911-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790257629041645-13911-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790257629041645-13911-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790257629041645-13911-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.149.254:34149
--webserver_interface=127.13.149.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.254.keytab
--principal=kudu/127.13.149.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.149.212:39827
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.149.254:34149
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790257629041645-13911-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790257629041645-13911-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790257629041645-13911-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 13:50:09.562108 27626 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 13:50:09.562338 27626 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 13:50:09.562371 27626 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 13:50:09.563536 27626 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 13:50:09.563580 27626 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 13:50:09.563598 27626 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 13:50:09.563614 27626 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 13:50:09.563632 27626 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 13:50:09.565140 27626 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.149.212:39827
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790257629041645-13911-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790257629041645-13911-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.149.254:34149
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.254.keytab
--principal=kudu/127.13.149.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.149.254:34149
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790257629041645-13911-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.149.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27626
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790257629041645-13911-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 12:29:37 UTC on ccycloud.quasar-iqzbxn.root.comops.site
build id 4670106
I20260924 13:50:09.565392 27626 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 13:50:09.565595 27626 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 13:50:09.570181 27634 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 13:50:09.570374 27626 server_base.cc:1034] running on GCE node
W20260924 13:50:09.570302 27632 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 13:50:09.570324 27631 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 13:50:09.570698 27626 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 13:50:09.570979 27626 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 13:50:09.572114 27626 hybrid_clock.cc:648] HybridClock initialized: now 1790257809572085 us; error 44 us; skew 500 ppm
Sep 24 13:50:09 dist-test-slave-sjl2 krb5kdc[27613](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257809, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.149.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 13:50:09.574548 27626 init.cc:377] Logged in from keytab as kudu/127.13.149.254@KRBTEST.COM (short username kudu)
I20260924 13:50:09.574960 27626 webserver.cc:460] Webserver started at http://127.13.149.254:33059/ using document root <none> and password file <none>
I20260924 13:50:09.575222 27626 fs_manager.cc:359] Metadata directory not provided
I20260924 13:50:09.575270 27626 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 13:50:09.575395 27626 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 13:50:09.576443 27626 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790257629041645-13911-0/minicluster-data/master-0/data/instance:
uuid: "5cde690887ba416e954be4f359d2d226"
format_stamp: "Formatted at 2026-09-24 13:50:09 on dist-test-slave-sjl2"
server_key: "8a05c0e39738e0d01f29fd1dd2cca810"
server_key_iv: "bfaadfa4250a03a24ae8f90d4aa5755b"
server_key_version: "encryptionkey@0"
I20260924 13:50:09.576764 27626 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790257629041645-13911-0/minicluster-data/master-0/wal/instance:
uuid: "5cde690887ba416e954be4f359d2d226"
format_stamp: "Formatted at 2026-09-24 13:50:09 on dist-test-slave-sjl2"
server_key: "8a05c0e39738e0d01f29fd1dd2cca810"
server_key_iv: "bfaadfa4250a03a24ae8f90d4aa5755b"
server_key_version: "encryptionkey@0"
I20260924 13:50:09.578070 27626 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260924 13:50:09.578817 27641 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 13:50:09.578963 27626 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260924 13:50:09.579046 27626 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790257629041645-13911-0/minicluster-data/master-0/data,/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790257629041645-13911-0/minicluster-data/master-0/wal
uuid: "5cde690887ba416e954be4f359d2d226"
format_stamp: "Formatted at 2026-09-24 13:50:09 on dist-test-slave-sjl2"
server_key: "8a05c0e39738e0d01f29fd1dd2cca810"
server_key_iv: "bfaadfa4250a03a24ae8f90d4aa5755b"
server_key_version: "encryptionkey@0"
I20260924 13:50:09.579108 27626 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790257629041645-13911-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790257629041645-13911-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790257629041645-13911-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 13:50:09.599197 27626 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 13:50:09.601282 27626 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 13:50:09.601413 27626 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 13:50:09.605118 27626 rpc_server.cc:307] RPC server started. Bound to: 127.13.149.254:34149
I20260924 13:50:09.605152 27693 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.149.254:34149 every 8 connection(s)
I20260924 13:50:09.605494 27626 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790257629041645-13911-0/minicluster-data/master-0/data/info.pb
I20260924 13:50:09.606333 27694 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 13:50:09.608713 27694 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5cde690887ba416e954be4f359d2d226: Bootstrap starting.
I20260924 13:50:09.608794 13911 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJN9WRF/build/release/bin/kudu as pid 27626
I20260924 13:50:09.608899 13911 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790257629041645-13911-0/minicluster-data/master-0/wal/instance
I20260924 13:50:09.609086 13911 external_mini_cluster.cc:1428] Setting key a02feac9bd12cafa3503d737f8e6823a
I20260924 13:50:09.609373 27694 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5cde690887ba416e954be4f359d2d226: Neither blocks nor log segments found. Creating new log.
I20260924 13:50:09.609699 27694 log.cc:824] T 00000000000000000000000000000000 P 5cde690887ba416e954be4f359d2d226: Log is configured to *not* fsync() on all Append() calls
I20260924 13:50:09.610404 27694 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5cde690887ba416e954be4f359d2d226: No bootstrap required, opened a new log
I20260924 13:50:09.612695 27694 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5cde690887ba416e954be4f359d2d226 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5cde690887ba416e954be4f359d2d226" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 34149 } }
I20260924 13:50:09.612852 27694 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5cde690887ba416e954be4f359d2d226 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 13:50:09.612876 27694 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5cde690887ba416e954be4f359d2d226 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cde690887ba416e954be4f359d2d226, State: Initialized, Role: FOLLOWER
I20260924 13:50:09.613011 27694 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5cde690887ba416e954be4f359d2d226 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5cde690887ba416e954be4f359d2d226" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 34149 } }
I20260924 13:50:09.613081 27694 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5cde690887ba416e954be4f359d2d226 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 13:50:09.613103 27694 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5cde690887ba416e954be4f359d2d226 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 13:50:09.613148 27694 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5cde690887ba416e954be4f359d2d226 [term 0 FOLLOWER]: Advancing to term 1
I20260924 13:50:09.613750 27694 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5cde690887ba416e954be4f359d2d226 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5cde690887ba416e954be4f359d2d226" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 34149 } }
I20260924 13:50:09.613853 27694 leader_election.cc:302] T 00000000000000000000000000000000 P 5cde690887ba416e954be4f359d2d226 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5cde690887ba416e954be4f359d2d226; no voters:
I20260924 13:50:09.614012 27694 leader_election.cc:288] T 00000000000000000000000000000000 P 5cde690887ba416e954be4f359d2d226 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 13:50:09.614048 27699 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5cde690887ba416e954be4f359d2d226 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 13:50:09.614236 27699 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5cde690887ba416e954be4f359d2d226 [term 1 LEADER]: Becoming Leader. State: Replica: 5cde690887ba416e954be4f359d2d226, State: Running, Role: LEADER
I20260924 13:50:09.614250 27694 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5cde690887ba416e954be4f359d2d226 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 13:50:09.614352 27699 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5cde690887ba416e954be4f359d2d226 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5cde690887ba416e954be4f359d2d226" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 34149 } }
I20260924 13:50:09.614660 27699 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5cde690887ba416e954be4f359d2d226 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5cde690887ba416e954be4f359d2d226" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5cde690887ba416e954be4f359d2d226" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 34149 } } }
I20260924 13:50:09.614725 27699 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5cde690887ba416e954be4f359d2d226 [sys.catalog]: This master's current role is: LEADER
I20260924 13:50:09.614758 27700 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5cde690887ba416e954be4f359d2d226 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5cde690887ba416e954be4f359d2d226. Latest consensus state: current_term: 1 leader_uuid: "5cde690887ba416e954be4f359d2d226" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5cde690887ba416e954be4f359d2d226" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 34149 } } }
I20260924 13:50:09.614835 27700 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5cde690887ba416e954be4f359d2d226 [sys.catalog]: This master's current role is: LEADER
W20260924 13:50:09.616055 27714 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 5cde690887ba416e954be4f359d2d226: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260924 13:50:09.616109 27714 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
Sep 24 13:50:09 dist-test-slave-sjl2 krb5kdc[27613](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257809, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.149.254@KRBTEST.COM
I20260924 13:50:09.616247 27705 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 13:50:09.616416 27705 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 13:50:09.618137 27705 catalog_manager.cc:1347] Generated new cluster ID: 73d604001ba043269d3f63ac374b1705
I20260924 13:50:09.618196 27705 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 13:50:09.620313 27697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:50:09.609611 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49932 (local address 127.13.149.254:34149)
0924 13:50:09.609807 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0924 13:50:09.609817 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0924 13:50:09.609857 (+ 40us) server_negotiation.cc:407] Connection header received
0924 13:50:09.610398 (+ 541us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:50:09.610407 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:50:09.610469 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:50:09.610616 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:50:09.611186 (+ 570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:50:09.612587 (+ 1401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:50:09.613450 (+ 863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:50:09.614064 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:50:09.614179 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:50:09.616719 (+ 2540us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:50:09.616723 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:50:09.616723 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 13:50:09.616736 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:50:09.618778 (+ 2042us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:50:09.619138 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:50:09.619141 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:50:09.619142 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:50:09.619248 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:50:09.619414 (+ 166us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:50:09.619415 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:50:09.619415 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:50:09.619732 (+ 317us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:50:09.619893 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0924 13:50:09.619967 (+ 74us) server_negotiation.cc:299] Negotiation successful
0924 13:50:09.620098 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":145,"thread_start_us":87,"threads_started":1}
I20260924 13:50:09.621209 13911 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260924 13:50:09.623647 27705 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 13:50:09.624339 27705 catalog_manager.cc:1504] Loading token signing keys...
I20260924 13:50:09.631181 27705 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5cde690887ba416e954be4f359d2d226: Generated new TSK 0
I20260924 13:50:09.631367 27705 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260924 13:50:09.675094 13911 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJN9WRF/build/release/bin/kudu with pid 27626
2026-09-24T13:50:09Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1266 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790257629041645-13911-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 13:50:09 dist-test-slave-sjl2 krb5kdc[27721](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 13:50:09 dist-test-slave-sjl2 krb5kdc[27721](info): set up 2 sockets
Sep 24 13:50:09 dist-test-slave-sjl2 krb5kdc[27721](info): commencing operation
krb5kdc: starting...
W20260924 13:50:10.621855 13911 mini_kdc.cc:121] Time spent starting KDC: real 0.929s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790257629041645-13911-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790257629041645-13911-0/krb5kdc/test-user.keytab.
Sep 24 13:50:10 dist-test-slave-sjl2 krb5kdc[27721](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257810, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T13:50:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T13:50:10Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.149.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.149.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.149.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.254.keytab.
Entry for principal kudu/127.13.149.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.254.keytab.
I20260924 13:50:10.738041 13911 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJN9WRF/build/release/bin/kudu
/tmp/dist-test-taskJN9WRF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790257629041645-13911-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790257629041645-13911-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790257629041645-13911-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790257629041645-13911-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.149.254:36075
--webserver_interface=127.13.149.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.254.keytab
--principal=kudu/127.13.149.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.149.212:45289
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.149.254:36075
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790257629041645-13911-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790257629041645-13911-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790257629041645-13911-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 13:50:10.817777 27734 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 13:50:10.817987 27734 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 13:50:10.818018 27734 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 13:50:10.819166 27734 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 13:50:10.819209 27734 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 13:50:10.819228 27734 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 13:50:10.819303 27734 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 13:50:10.819324 27734 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 13:50:10.820813 27734 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.149.212:45289
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790257629041645-13911-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790257629041645-13911-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.149.254:36075
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.254.keytab
--principal=kudu/127.13.149.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.149.254:36075
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790257629041645-13911-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.149.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27734
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790257629041645-13911-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 12:29:37 UTC on ccycloud.quasar-iqzbxn.root.comops.site
build id 4670106
I20260924 13:50:10.821089 27734 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 13:50:10.821298 27734 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 13:50:10.825960 27740 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 13:50:10.825955 27742 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 13:50:10.825960 27739 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 13:50:10.826099 27734 server_base.cc:1034] running on GCE node
I20260924 13:50:10.826399 27734 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 13:50:10.826673 27734 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 13:50:10.827811 27734 hybrid_clock.cc:648] HybridClock initialized: now 1790257810827793 us; error 36 us; skew 500 ppm
Sep 24 13:50:10 dist-test-slave-sjl2 krb5kdc[27721](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257810, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.149.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 13:50:10.830192 27734 init.cc:377] Logged in from keytab as kudu/127.13.149.254@KRBTEST.COM (short username kudu)
I20260924 13:50:10.830615 27734 webserver.cc:460] Webserver started at http://127.13.149.254:44463/ using document root <none> and password file <none>
I20260924 13:50:10.830798 27734 fs_manager.cc:359] Metadata directory not provided
I20260924 13:50:10.830837 27734 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 13:50:10.830931 27734 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 13:50:10.831928 27734 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790257629041645-13911-0/minicluster-data/master-0/data/instance:
uuid: "ffaff00f2fee4d03bb1f71c504fabd7f"
format_stamp: "Formatted at 2026-09-24 13:50:10 on dist-test-slave-sjl2"
server_key: "b5ae1bf96c07654927cebe93e7a64d6e"
server_key_iv: "0efeda2723646de6a9a598a3753139cd"
server_key_version: "encryptionkey@0"
I20260924 13:50:10.832216 27734 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790257629041645-13911-0/minicluster-data/master-0/wal/instance:
uuid: "ffaff00f2fee4d03bb1f71c504fabd7f"
format_stamp: "Formatted at 2026-09-24 13:50:10 on dist-test-slave-sjl2"
server_key: "b5ae1bf96c07654927cebe93e7a64d6e"
server_key_iv: "0efeda2723646de6a9a598a3753139cd"
server_key_version: "encryptionkey@0"
I20260924 13:50:10.833562 27734 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260924 13:50:10.834386 27749 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 13:50:10.834576 27734 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260924 13:50:10.834673 27734 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790257629041645-13911-0/minicluster-data/master-0/data,/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790257629041645-13911-0/minicluster-data/master-0/wal
uuid: "ffaff00f2fee4d03bb1f71c504fabd7f"
format_stamp: "Formatted at 2026-09-24 13:50:10 on dist-test-slave-sjl2"
server_key: "b5ae1bf96c07654927cebe93e7a64d6e"
server_key_iv: "0efeda2723646de6a9a598a3753139cd"
server_key_version: "encryptionkey@0"
I20260924 13:50:10.834748 27734 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790257629041645-13911-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790257629041645-13911-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790257629041645-13911-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 13:50:10.848285 27734 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 13:50:10.850404 27734 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 13:50:10.850526 27734 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 13:50:10.854305 27734 rpc_server.cc:307] RPC server started. Bound to: 127.13.149.254:36075
I20260924 13:50:10.854349 27801 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.149.254:36075 every 8 connection(s)
I20260924 13:50:10.854653 27734 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790257629041645-13911-0/minicluster-data/master-0/data/info.pb
I20260924 13:50:10.855607 27802 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 13:50:10.857954 27802 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ffaff00f2fee4d03bb1f71c504fabd7f: Bootstrap starting.
I20260924 13:50:10.858498 27802 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ffaff00f2fee4d03bb1f71c504fabd7f: Neither blocks nor log segments found. Creating new log.
I20260924 13:50:10.858729 27802 log.cc:824] T 00000000000000000000000000000000 P ffaff00f2fee4d03bb1f71c504fabd7f: Log is configured to *not* fsync() on all Append() calls
I20260924 13:50:10.859438 27802 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ffaff00f2fee4d03bb1f71c504fabd7f: No bootstrap required, opened a new log
I20260924 13:50:10.860997 27802 raft_consensus.cc:348] T 00000000000000000000000000000000 P ffaff00f2fee4d03bb1f71c504fabd7f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ffaff00f2fee4d03bb1f71c504fabd7f" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 36075 } }
I20260924 13:50:10.861136 27802 raft_consensus.cc:374] T 00000000000000000000000000000000 P ffaff00f2fee4d03bb1f71c504fabd7f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 13:50:10.861156 27802 raft_consensus.cc:729] T 00000000000000000000000000000000 P ffaff00f2fee4d03bb1f71c504fabd7f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ffaff00f2fee4d03bb1f71c504fabd7f, State: Initialized, Role: FOLLOWER
I20260924 13:50:10.861269 27802 consensus_queue.cc:260] T 00000000000000000000000000000000 P ffaff00f2fee4d03bb1f71c504fabd7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ffaff00f2fee4d03bb1f71c504fabd7f" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 36075 } }
I20260924 13:50:10.861320 27802 raft_consensus.cc:388] T 00000000000000000000000000000000 P ffaff00f2fee4d03bb1f71c504fabd7f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 13:50:10.861347 27802 raft_consensus.cc:482] T 00000000000000000000000000000000 P ffaff00f2fee4d03bb1f71c504fabd7f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 13:50:10.861377 27802 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ffaff00f2fee4d03bb1f71c504fabd7f [term 0 FOLLOWER]: Advancing to term 1
I20260924 13:50:10.862128 27802 raft_consensus.cc:504] T 00000000000000000000000000000000 P ffaff00f2fee4d03bb1f71c504fabd7f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ffaff00f2fee4d03bb1f71c504fabd7f" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 36075 } }
I20260924 13:50:10.862216 27802 leader_election.cc:302] T 00000000000000000000000000000000 P ffaff00f2fee4d03bb1f71c504fabd7f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ffaff00f2fee4d03bb1f71c504fabd7f; no voters:
I20260924 13:50:10.862339 27802 leader_election.cc:288] T 00000000000000000000000000000000 P ffaff00f2fee4d03bb1f71c504fabd7f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 13:50:10.862386 27805 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ffaff00f2fee4d03bb1f71c504fabd7f [term 1 FOLLOWER]: Leader election won for term 1
I20260924 13:50:10.862557 27805 raft_consensus.cc:686] T 00000000000000000000000000000000 P ffaff00f2fee4d03bb1f71c504fabd7f [term 1 LEADER]: Becoming Leader. State: Replica: ffaff00f2fee4d03bb1f71c504fabd7f, State: Running, Role: LEADER
I20260924 13:50:10.862560 27802 sys_catalog.cc:564] T 00000000000000000000000000000000 P ffaff00f2fee4d03bb1f71c504fabd7f [sys.catalog]: configured and running, proceeding with master startup.
I20260924 13:50:10.862686 27805 consensus_queue.cc:237] T 00000000000000000000000000000000 P ffaff00f2fee4d03bb1f71c504fabd7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ffaff00f2fee4d03bb1f71c504fabd7f" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 36075 } }
I20260924 13:50:10.862887 13911 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJN9WRF/build/release/bin/kudu as pid 27734
I20260924 13:50:10.862991 13911 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790257629041645-13911-0/minicluster-data/master-0/wal/instance
I20260924 13:50:10.863217 13911 external_mini_cluster.cc:1428] Setting key 9f8431d3462d4f630de494b9cd8c6744
I20260924 13:50:10.863418 27807 sys_catalog.cc:455] T 00000000000000000000000000000000 P ffaff00f2fee4d03bb1f71c504fabd7f [sys.catalog]: SysCatalogTable state changed. Reason: New leader ffaff00f2fee4d03bb1f71c504fabd7f. Latest consensus state: current_term: 1 leader_uuid: "ffaff00f2fee4d03bb1f71c504fabd7f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ffaff00f2fee4d03bb1f71c504fabd7f" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 36075 } } }
I20260924 13:50:10.863497 27807 sys_catalog.cc:458] T 00000000000000000000000000000000 P ffaff00f2fee4d03bb1f71c504fabd7f [sys.catalog]: This master's current role is: LEADER
I20260924 13:50:10.863866 27814 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 13:50:10.864015 27806 sys_catalog.cc:455] T 00000000000000000000000000000000 P ffaff00f2fee4d03bb1f71c504fabd7f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ffaff00f2fee4d03bb1f71c504fabd7f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ffaff00f2fee4d03bb1f71c504fabd7f" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 36075 } } }
I20260924 13:50:10.864210 27806 sys_catalog.cc:458] T 00000000000000000000000000000000 P ffaff00f2fee4d03bb1f71c504fabd7f [sys.catalog]: This master's current role is: LEADER
I20260924 13:50:10.864517 27814 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 13:50:10.866047 27814 catalog_manager.cc:1347] Generated new cluster ID: da0001853c6b4ac9b2a8cdec412b8ea4
I20260924 13:50:10.866103 27814 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 24 13:50:10 dist-test-slave-sjl2 krb5kdc[27721](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257810, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.149.254@KRBTEST.COM
I20260924 13:50:10.880715 27823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:50:10.863873 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48698 (local address 127.13.149.254:36075)
0924 13:50:10.864322 (+ 449us) server_negotiation.cc:206] Beginning negotiation
0924 13:50:10.864333 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0924 13:50:10.864479 (+ 146us) server_negotiation.cc:407] Connection header received
0924 13:50:10.864620 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:50:10.864627 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:50:10.864693 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:50:10.864810 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:50:10.865658 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:50:10.873782 (+ 8124us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:50:10.874726 (+ 944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:50:10.875201 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:50:10.875302 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:50:10.877612 (+ 2310us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:50:10.877618 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:50:10.877618 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 13:50:10.877631 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:50:10.879176 (+ 1545us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:50:10.879501 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:50:10.879504 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:50:10.879505 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:50:10.879595 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:50:10.879770 (+ 175us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:50:10.879771 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:50:10.879771 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:50:10.880017 (+ 246us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:50:10.880162 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0924 13:50:10.880269 (+ 107us) server_negotiation.cc:299] Negotiation successful
0924 13:50:10.880522 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":402,"thread_start_us":65,"threads_started":1}
I20260924 13:50:10.896689 27814 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 13:50:10.897301 27814 catalog_manager.cc:1504] Loading token signing keys...
I20260924 13:50:10.906985 27814 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ffaff00f2fee4d03bb1f71c504fabd7f: Generated new TSK 0
I20260924 13:50:10.907243 27814 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.13.149.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.149.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.149.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.193.keytab.
Entry for principal kudu/127.13.149.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.193.keytab.
I20260924 13:50:10.939561 13911 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJN9WRF/build/release/bin/kudu
/tmp/dist-test-taskJN9WRF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790257629041645-13911-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790257629041645-13911-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790257629041645-13911-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790257629041645-13911-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.149.193:0
--local_ip_for_outbound_sockets=127.13.149.193
--webserver_interface=127.13.149.193
--webserver_port=0
--tserver_master_addrs=127.13.149.254:36075
--keytab_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.193.keytab
--principal=kudu/127.13.149.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.149.212:45289
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790257629041645-13911-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790257629041645-13911-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790257629041645-13911-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 13:50:11.028213 27828 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 13:50:11.028429 27828 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 13:50:11.028453 27828 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 13:50:11.029706 27828 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 13:50:11.029752 27828 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 13:50:11.029778 27828 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.149.193
I20260924 13:50:11.031569 27828 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.149.212:45289
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790257629041645-13911-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790257629041645-13911-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.193.keytab
--principal=kudu/127.13.149.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.149.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790257629041645-13911-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.149.193
--webserver_port=0
--tserver_master_addrs=127.13.149.254:36075
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27828
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.149.193
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790257629041645-13911-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 12:29:37 UTC on ccycloud.quasar-iqzbxn.root.comops.site
build id 4670106
I20260924 13:50:11.031816 27828 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 13:50:11.032032 27828 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 13:50:11.036791 27833 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 13:50:11.036801 27834 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 13:50:11.036887 27828 server_base.cc:1034] running on GCE node
W20260924 13:50:11.037032 27836 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 13:50:11.037269 27828 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 13:50:11.037528 27828 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 13:50:11.038663 27828 hybrid_clock.cc:648] HybridClock initialized: now 1790257811038648 us; error 32 us; skew 500 ppm
Sep 24 13:50:11 dist-test-slave-sjl2 krb5kdc[27721](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257811, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.149.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 13:50:11.041143 27828 init.cc:377] Logged in from keytab as kudu/127.13.149.193@KRBTEST.COM (short username kudu)
I20260924 13:50:11.041538 27828 webserver.cc:460] Webserver started at http://127.13.149.193:43713/ using document root <none> and password file <none>
I20260924 13:50:11.041725 27828 fs_manager.cc:359] Metadata directory not provided
I20260924 13:50:11.041764 27828 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 13:50:11.041858 27828 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 13:50:11.042876 27828 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790257629041645-13911-0/minicluster-data/ts-0/data/instance:
uuid: "46b4fd024b654a5497f77167a74589cf"
format_stamp: "Formatted at 2026-09-24 13:50:11 on dist-test-slave-sjl2"
server_key: "421ffd5752f236310d95d01d0d561f72"
server_key_iv: "3279b4f8ab63fc4df9ce5e8ad6474835"
server_key_version: "encryptionkey@0"
I20260924 13:50:11.043166 27828 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790257629041645-13911-0/minicluster-data/ts-0/wal/instance:
uuid: "46b4fd024b654a5497f77167a74589cf"
format_stamp: "Formatted at 2026-09-24 13:50:11 on dist-test-slave-sjl2"
server_key: "421ffd5752f236310d95d01d0d561f72"
server_key_iv: "3279b4f8ab63fc4df9ce5e8ad6474835"
server_key_version: "encryptionkey@0"
I20260924 13:50:11.044335 27828 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260924 13:50:11.045027 27843 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 13:50:11.045153 27828 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 13:50:11.045234 27828 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790257629041645-13911-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790257629041645-13911-0/minicluster-data/ts-0/wal
uuid: "46b4fd024b654a5497f77167a74589cf"
format_stamp: "Formatted at 2026-09-24 13:50:11 on dist-test-slave-sjl2"
server_key: "421ffd5752f236310d95d01d0d561f72"
server_key_iv: "3279b4f8ab63fc4df9ce5e8ad6474835"
server_key_version: "encryptionkey@0"
I20260924 13:50:11.045300 27828 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790257629041645-13911-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790257629041645-13911-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790257629041645-13911-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 13:50:11.053340 27828 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 13:50:11.055310 27828 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 13:50:11.055433 27828 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 13:50:11.055629 27828 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 13:50:11.055909 27828 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 13:50:11.055948 27828 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 13:50:11.055969 27828 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 13:50:11.055992 27828 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 13:50:11.061479 27828 rpc_server.cc:307] RPC server started. Bound to: 127.13.149.193:43429
I20260924 13:50:11.061539 27956 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.149.193:43429 every 8 connection(s)
I20260924 13:50:11.061828 27828 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790257629041645-13911-0/minicluster-data/ts-0/data/info.pb
I20260924 13:50:11.064764 13911 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJN9WRF/build/release/bin/kudu as pid 27828
I20260924 13:50:11.064860 13911 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790257629041645-13911-0/minicluster-data/ts-0/wal/instance
I20260924 13:50:11.065088 13911 external_mini_cluster.cc:1428] Setting key 6835d77d78d81c1b27bffa37277c3558
Sep 24 13:50:11 dist-test-slave-sjl2 krb5kdc[27721](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257811, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.149.193@KRBTEST.COM for kudu/127.13.149.254@KRBTEST.COM
I20260924 13:50:11.071146 27823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:50:11.062475 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.149.193:45009 (local address 127.13.149.254:36075)
0924 13:50:11.062521 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0924 13:50:11.062523 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:50:11.062728 (+ 205us) server_negotiation.cc:407] Connection header received
0924 13:50:11.063212 (+ 484us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:50:11.063213 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:50:11.063224 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:50:11.063287 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:50:11.064318 (+ 1031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:50:11.064877 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:50:11.066040 (+ 1163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:50:11.066228 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:50:11.066285 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:50:11.068467 (+ 2182us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:50:11.068468 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:50:11.068468 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 13:50:11.068475 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:50:11.069759 (+ 1284us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:50:11.070218 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:50:11.070219 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:50:11.070219 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:50:11.070289 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:50:11.070545 (+ 256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:50:11.070546 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:50:11.070546 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:50:11.070644 (+ 98us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:50:11.070689 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0924 13:50:11.071008 (+ 319us) server_negotiation.cc:299] Negotiation successful
0924 13:50:11.071076 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260924 13:50:11.071405 27959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:50:11.062466 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:36075 (local address 127.13.149.193:45009)
0924 13:50:11.062612 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0924 13:50:11.062646 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0924 13:50:11.063125 (+ 479us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:50:11.063407 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:50:11.063410 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:50:11.064023 (+ 613us) client_negotiation.cc:189] Negotiated authn=SASL
0924 13:50:11.064227 (+ 204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:50:11.064234 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:50:11.064998 (+ 764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:50:11.064999 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:50:11.065966 (+ 967us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:50:11.065969 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:50:11.066312 (+ 343us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:50:11.066313 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:50:11.066455 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:50:11.066939 (+ 484us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 13:50:11.066948 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0924 13:50:11.068367 (+ 1419us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 13:50:11.069840 (+ 1473us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 13:50:11.069848 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 13:50:11.069851 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0924 13:50:11.070147 (+ 296us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 13:50:11.070348 (+ 201us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 13:50:11.070349 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 13:50:11.070349 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 13:50:11.070486 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 13:50:11.070704 (+ 218us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 13:50:11.070710 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 13:50:11.070877 (+ 167us) client_negotiation.cc:769] Sending connection context
0924 13:50:11.070997 (+ 120us) client_negotiation.cc:240] Negotiation successful
0924 13:50:11.071169 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":79,"threads_started":1}
I20260924 13:50:11.071969 27957 heartbeater.cc:347] Connected to a master server at 127.13.149.254:36075
I20260924 13:50:11.072059 27957 heartbeater.cc:464] Registering TS with master...
I20260924 13:50:11.072235 27957 heartbeater.cc:511] Master 127.13.149.254:36075 requested a full tablet report, sending...
I20260924 13:50:11.072732 27766 ts_manager.cc:194] Registered new tserver with Master: 46b4fd024b654a5497f77167a74589cf (127.13.149.193:43429)
WARNING: no policy specified for kudu/127.13.149.194@KRBTEST.COM; defaulting to no policy
I20260924 13:50:11.073388 27766 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.149.193@KRBTEST.COM'} at 127.13.149.193:45009
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.149.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.149.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.194.keytab.
Entry for principal kudu/127.13.149.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.194.keytab.
I20260924 13:50:11.091252 13911 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJN9WRF/build/release/bin/kudu
/tmp/dist-test-taskJN9WRF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790257629041645-13911-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790257629041645-13911-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790257629041645-13911-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790257629041645-13911-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.149.194:0
--local_ip_for_outbound_sockets=127.13.149.194
--webserver_interface=127.13.149.194
--webserver_port=0
--tserver_master_addrs=127.13.149.254:36075
--keytab_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.194.keytab
--principal=kudu/127.13.149.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.149.212:45289
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790257629041645-13911-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790257629041645-13911-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790257629041645-13911-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 13:50:11.171559 27962 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 13:50:11.171788 27962 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 13:50:11.171823 27962 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 13:50:11.173036 27962 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 13:50:11.173075 27962 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 13:50:11.173112 27962 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.149.194
I20260924 13:50:11.174626 27962 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.149.212:45289
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790257629041645-13911-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790257629041645-13911-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.194.keytab
--principal=kudu/127.13.149.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.149.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790257629041645-13911-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.149.194
--webserver_port=0
--tserver_master_addrs=127.13.149.254:36075
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27962
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.149.194
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790257629041645-13911-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 12:29:37 UTC on ccycloud.quasar-iqzbxn.root.comops.site
build id 4670106
I20260924 13:50:11.174861 27962 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 13:50:11.175067 27962 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 13:50:11.179939 27970 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 13:50:11.179999 27967 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 13:50:11.179992 27962 server_base.cc:1034] running on GCE node
W20260924 13:50:11.179967 27968 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 13:50:11.180341 27962 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 13:50:11.180564 27962 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 13:50:11.181715 27962 hybrid_clock.cc:648] HybridClock initialized: now 1790257811181698 us; error 35 us; skew 500 ppm
Sep 24 13:50:11 dist-test-slave-sjl2 krb5kdc[27721](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257811, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.149.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 13:50:11.184072 27962 init.cc:377] Logged in from keytab as kudu/127.13.149.194@KRBTEST.COM (short username kudu)
I20260924 13:50:11.184471 27962 webserver.cc:460] Webserver started at http://127.13.149.194:42307/ using document root <none> and password file <none>
I20260924 13:50:11.184652 27962 fs_manager.cc:359] Metadata directory not provided
I20260924 13:50:11.184698 27962 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 13:50:11.184788 27962 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 13:50:11.185843 27962 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790257629041645-13911-0/minicluster-data/ts-1/data/instance:
uuid: "666006d5e083435b9d59291a298b266c"
format_stamp: "Formatted at 2026-09-24 13:50:11 on dist-test-slave-sjl2"
server_key: "44b1ebca15f1cd56048d3c315cd6857a"
server_key_iv: "a09f710677d3ee077cb6294c3dec6b22"
server_key_version: "encryptionkey@0"
I20260924 13:50:11.186136 27962 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790257629041645-13911-0/minicluster-data/ts-1/wal/instance:
uuid: "666006d5e083435b9d59291a298b266c"
format_stamp: "Formatted at 2026-09-24 13:50:11 on dist-test-slave-sjl2"
server_key: "44b1ebca15f1cd56048d3c315cd6857a"
server_key_iv: "a09f710677d3ee077cb6294c3dec6b22"
server_key_version: "encryptionkey@0"
I20260924 13:50:11.187343 27962 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260924 13:50:11.188014 27977 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 13:50:11.188156 27962 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 13:50:11.188241 27962 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790257629041645-13911-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790257629041645-13911-0/minicluster-data/ts-1/wal
uuid: "666006d5e083435b9d59291a298b266c"
format_stamp: "Formatted at 2026-09-24 13:50:11 on dist-test-slave-sjl2"
server_key: "44b1ebca15f1cd56048d3c315cd6857a"
server_key_iv: "a09f710677d3ee077cb6294c3dec6b22"
server_key_version: "encryptionkey@0"
I20260924 13:50:11.188308 27962 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790257629041645-13911-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790257629041645-13911-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790257629041645-13911-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 13:50:11.211324 27962 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 13:50:11.213279 27962 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 13:50:11.213390 27962 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 13:50:11.213575 27962 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 13:50:11.213876 27962 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 13:50:11.213915 27962 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 13:50:11.213938 27962 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 13:50:11.213961 27962 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 13:50:11.219442 27962 rpc_server.cc:307] RPC server started. Bound to: 127.13.149.194:45113
I20260924 13:50:11.219521 28090 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.149.194:45113 every 8 connection(s)
I20260924 13:50:11.219807 27962 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790257629041645-13911-0/minicluster-data/ts-1/data/info.pb
Sep 24 13:50:11 dist-test-slave-sjl2 krb5kdc[27721](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257811, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.149.194@KRBTEST.COM for kudu/127.13.149.254@KRBTEST.COM
I20260924 13:50:11.226060 13911 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJN9WRF/build/release/bin/kudu as pid 27962
I20260924 13:50:11.226202 13911 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790257629041645-13911-0/minicluster-data/ts-1/wal/instance
I20260924 13:50:11.226377 13911 external_mini_cluster.cc:1428] Setting key 6e9bc1e03fdbe77c2ea7161b76fcaf50
I20260924 13:50:11.228909 27823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:50:11.220490 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.149.194:39395 (local address 127.13.149.254:36075)
0924 13:50:11.220536 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0924 13:50:11.220538 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:50:11.220746 (+ 208us) server_negotiation.cc:407] Connection header received
0924 13:50:11.221312 (+ 566us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:50:11.221313 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:50:11.221323 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:50:11.221365 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:50:11.222263 (+ 898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:50:11.222796 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:50:11.223774 (+ 978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:50:11.223956 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:50:11.224003 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:50:11.226100 (+ 2097us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:50:11.226102 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:50:11.226102 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 13:50:11.226109 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:50:11.227531 (+ 1422us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:50:11.227980 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:50:11.227981 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:50:11.227982 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:50:11.228038 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:50:11.228294 (+ 256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:50:11.228295 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:50:11.228295 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:50:11.228437 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:50:11.228501 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0924 13:50:11.228732 (+ 231us) server_negotiation.cc:299] Negotiation successful
0924 13:50:11.228816 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260924 13:50:11.229058 28093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:50:11.220457 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:36075 (local address 127.13.149.194:39395)
0924 13:50:11.220624 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0924 13:50:11.220667 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0924 13:50:11.221181 (+ 514us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:50:11.221419 (+ 238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:50:11.221423 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:50:11.221976 (+ 553us) client_negotiation.cc:189] Negotiated authn=SASL
0924 13:50:11.222181 (+ 205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:50:11.222186 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:50:11.222880 (+ 694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:50:11.222881 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:50:11.223707 (+ 826us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:50:11.223709 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:50:11.224052 (+ 343us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:50:11.224053 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:50:11.224175 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:50:11.224648 (+ 473us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 13:50:11.224656 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0924 13:50:11.226006 (+ 1350us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 13:50:11.227613 (+ 1607us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 13:50:11.227621 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 13:50:11.227623 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 13:50:11.227905 (+ 282us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 13:50:11.228088 (+ 183us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 13:50:11.228089 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 13:50:11.228089 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 13:50:11.228220 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 13:50:11.228496 (+ 276us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 13:50:11.228500 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 13:50:11.228619 (+ 119us) client_negotiation.cc:769] Sending connection context
0924 13:50:11.228702 (+ 83us) client_negotiation.cc:240] Negotiation successful
0924 13:50:11.228827 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":69,"threads_started":1}
I20260924 13:50:11.229660 28091 heartbeater.cc:347] Connected to a master server at 127.13.149.254:36075
I20260924 13:50:11.229754 28091 heartbeater.cc:464] Registering TS with master...
I20260924 13:50:11.229923 28091 heartbeater.cc:511] Master 127.13.149.254:36075 requested a full tablet report, sending...
I20260924 13:50:11.230283 27766 ts_manager.cc:194] Registered new tserver with Master: 666006d5e083435b9d59291a298b266c (127.13.149.194:45113)
I20260924 13:50:11.230788 27766 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.149.194@KRBTEST.COM'} at 127.13.149.194:39395
WARNING: no policy specified for kudu/127.13.149.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.149.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.149.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.195.keytab.
Entry for principal kudu/127.13.149.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.195.keytab.
I20260924 13:50:11.251812 13911 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJN9WRF/build/release/bin/kudu
/tmp/dist-test-taskJN9WRF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790257629041645-13911-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790257629041645-13911-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790257629041645-13911-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790257629041645-13911-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.149.195:0
--local_ip_for_outbound_sockets=127.13.149.195
--webserver_interface=127.13.149.195
--webserver_port=0
--tserver_master_addrs=127.13.149.254:36075
--keytab_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.195.keytab
--principal=kudu/127.13.149.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.149.212:45289
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790257629041645-13911-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790257629041645-13911-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790257629041645-13911-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 13:50:11.329684 28096 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 13:50:11.329887 28096 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 13:50:11.329919 28096 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 13:50:11.331063 28096 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 13:50:11.331103 28096 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 13:50:11.331144 28096 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.149.195
I20260924 13:50:11.332582 28096 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.149.212:45289
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790257629041645-13911-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790257629041645-13911-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.195.keytab
--principal=kudu/127.13.149.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.149.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790257629041645-13911-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.149.195
--webserver_port=0
--tserver_master_addrs=127.13.149.254:36075
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28096
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.149.195
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790257629041645-13911-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 12:29:37 UTC on ccycloud.quasar-iqzbxn.root.comops.site
build id 4670106
I20260924 13:50:11.332806 28096 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 13:50:11.333037 28096 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 13:50:11.337759 28102 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 13:50:11.337774 28104 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 13:50:11.337774 28101 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 13:50:11.337982 28096 server_base.cc:1034] running on GCE node
I20260924 13:50:11.338212 28096 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 13:50:11.338454 28096 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 13:50:11.339594 28096 hybrid_clock.cc:648] HybridClock initialized: now 1790257811339580 us; error 26 us; skew 500 ppm
Sep 24 13:50:11 dist-test-slave-sjl2 krb5kdc[27721](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257811, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.149.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 13:50:11.341989 28096 init.cc:377] Logged in from keytab as kudu/127.13.149.195@KRBTEST.COM (short username kudu)
I20260924 13:50:11.342410 28096 webserver.cc:460] Webserver started at http://127.13.149.195:41563/ using document root <none> and password file <none>
I20260924 13:50:11.342621 28096 fs_manager.cc:359] Metadata directory not provided
I20260924 13:50:11.342676 28096 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 13:50:11.342799 28096 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 13:50:11.343832 28096 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790257629041645-13911-0/minicluster-data/ts-2/data/instance:
uuid: "8423e2366dc04ab8b01dc9d932e8ee75"
format_stamp: "Formatted at 2026-09-24 13:50:11 on dist-test-slave-sjl2"
server_key: "83f8cad45ec6d96e29788656175a6647"
server_key_iv: "6b65e4d52a42cc1cdac48a416fe21a38"
server_key_version: "encryptionkey@0"
I20260924 13:50:11.344144 28096 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790257629041645-13911-0/minicluster-data/ts-2/wal/instance:
uuid: "8423e2366dc04ab8b01dc9d932e8ee75"
format_stamp: "Formatted at 2026-09-24 13:50:11 on dist-test-slave-sjl2"
server_key: "83f8cad45ec6d96e29788656175a6647"
server_key_iv: "6b65e4d52a42cc1cdac48a416fe21a38"
server_key_version: "encryptionkey@0"
I20260924 13:50:11.345398 28096 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260924 13:50:11.346050 28111 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 13:50:11.346189 28096 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260924 13:50:11.346267 28096 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790257629041645-13911-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790257629041645-13911-0/minicluster-data/ts-2/wal
uuid: "8423e2366dc04ab8b01dc9d932e8ee75"
format_stamp: "Formatted at 2026-09-24 13:50:11 on dist-test-slave-sjl2"
server_key: "83f8cad45ec6d96e29788656175a6647"
server_key_iv: "6b65e4d52a42cc1cdac48a416fe21a38"
server_key_version: "encryptionkey@0"
I20260924 13:50:11.346326 28096 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790257629041645-13911-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790257629041645-13911-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790257629041645-13911-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 13:50:11.378885 28096 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 13:50:11.381062 28096 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 13:50:11.381211 28096 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 13:50:11.381434 28096 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 13:50:11.381724 28096 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 13:50:11.381757 28096 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 13:50:11.381786 28096 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 13:50:11.381809 28096 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 13:50:11.387390 28096 rpc_server.cc:307] RPC server started. Bound to: 127.13.149.195:45095
I20260924 13:50:11.387434 28224 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.149.195:45095 every 8 connection(s)
I20260924 13:50:11.387737 28096 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790257629041645-13911-0/minicluster-data/ts-2/data/info.pb
Sep 24 13:50:11 dist-test-slave-sjl2 krb5kdc[27721](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257811, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.149.195@KRBTEST.COM for kudu/127.13.149.254@KRBTEST.COM
I20260924 13:50:11.396370 27823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:50:11.388414 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.149.195:57081 (local address 127.13.149.254:36075)
0924 13:50:11.388492 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0924 13:50:11.388494 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:50:11.388657 (+ 163us) server_negotiation.cc:407] Connection header received
0924 13:50:11.389227 (+ 570us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:50:11.389228 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:50:11.389242 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:50:11.389293 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:50:11.390229 (+ 936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:50:11.390722 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:50:11.391775 (+ 1053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:50:11.391961 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:50:11.392040 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:50:11.394078 (+ 2038us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:50:11.394080 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:50:11.394080 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 13:50:11.394088 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:50:11.395138 (+ 1050us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:50:11.395499 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:50:11.395500 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:50:11.395501 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:50:11.395553 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:50:11.395768 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:50:11.395769 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:50:11.395769 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:50:11.395881 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:50:11.395937 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0924 13:50:11.396194 (+ 257us) server_negotiation.cc:299] Negotiation successful
0924 13:50:11.396276 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260924 13:50:11.396605 28227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:50:11.388387 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:36075 (local address 127.13.149.195:57081)
0924 13:50:11.388532 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0924 13:50:11.388570 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0924 13:50:11.389091 (+ 521us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:50:11.389361 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:50:11.389365 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:50:11.389937 (+ 572us) client_negotiation.cc:189] Negotiated authn=SASL
0924 13:50:11.390133 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:50:11.390138 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:50:11.390823 (+ 685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:50:11.390824 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:50:11.391687 (+ 863us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:50:11.391691 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:50:11.392049 (+ 358us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:50:11.392050 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:50:11.392172 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:50:11.392639 (+ 467us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 13:50:11.392647 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0924 13:50:11.393993 (+ 1346us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 13:50:11.395209 (+ 1216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 13:50:11.395216 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 13:50:11.395218 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 13:50:11.395434 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 13:50:11.395606 (+ 172us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 13:50:11.395607 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 13:50:11.395608 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 13:50:11.395707 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 13:50:11.395936 (+ 229us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 13:50:11.395943 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 13:50:11.396075 (+ 132us) client_negotiation.cc:769] Sending connection context
0924 13:50:11.396163 (+ 88us) client_negotiation.cc:240] Negotiation successful
0924 13:50:11.396333 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":71,"threads_started":1}
I20260924 13:50:11.396613 13911 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJN9WRF/build/release/bin/kudu as pid 28096
I20260924 13:50:11.396692 13911 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790257629041645-13911-0/minicluster-data/ts-2/wal/instance
I20260924 13:50:11.396876 13911 external_mini_cluster.cc:1428] Setting key a9d2e0fe74ecf3440352ac7c3d704c6d
I20260924 13:50:11.397064 28225 heartbeater.cc:347] Connected to a master server at 127.13.149.254:36075
I20260924 13:50:11.397169 28225 heartbeater.cc:464] Registering TS with master...
I20260924 13:50:11.397363 28225 heartbeater.cc:511] Master 127.13.149.254:36075 requested a full tablet report, sending...
I20260924 13:50:11.397735 27766 ts_manager.cc:194] Registered new tserver with Master: 8423e2366dc04ab8b01dc9d932e8ee75 (127.13.149.195:45095)
I20260924 13:50:11.398108 27766 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.149.195@KRBTEST.COM'} at 127.13.149.195:57081
I20260924 13:50:11.408680 13911 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260924 13:50:11.415867 27823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:50:11.410058 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48708 (local address 127.13.149.254:36075)
0924 13:50:11.410106 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0924 13:50:11.410108 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:50:11.410205 (+ 97us) server_negotiation.cc:407] Connection header received
0924 13:50:11.410277 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:50:11.410278 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:50:11.410289 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:50:11.410339 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:50:11.410897 (+ 558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:50:11.411386 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:50:11.412068 (+ 682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:50:11.412263 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:50:11.412316 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:50:11.413553 (+ 1237us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:50:11.413555 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:50:11.413555 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 13:50:11.413563 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:50:11.414775 (+ 1212us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:50:11.415082 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:50:11.415083 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:50:11.415084 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:50:11.415138 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:50:11.415303 (+ 165us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:50:11.415305 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:50:11.415305 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:50:11.415466 (+ 161us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:50:11.415560 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0924 13:50:11.415681 (+ 121us) server_negotiation.cc:299] Negotiation successful
0924 13:50:11.415774 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260924 13:50:11.417685 27766 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48708:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 13:50:11.419299 27766 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 13:50:11.425943 28238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:50:11.421994 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54614 (local address 127.13.149.195:45095)
0924 13:50:11.422170 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0924 13:50:11.422175 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 13:50:11.422347 (+ 172us) server_negotiation.cc:407] Connection header received
0924 13:50:11.422422 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:50:11.422424 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:50:11.422441 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:50:11.422527 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 13:50:11.422730 (+ 203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:50:11.423827 (+ 1097us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:50:11.425057 (+ 1230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:50:11.425547 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:50:11.425622 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:50:11.425680 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0924 13:50:11.425816 (+ 136us) server_negotiation.cc:299] Negotiation successful
0924 13:50:11.425857 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130,"thread_start_us":83,"threads_started":1}
I20260924 13:50:11.425961 28240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:50:11.421889 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.195:45095 (local address 127.0.0.1:54614)
0924 13:50:11.422273 (+ 384us) negotiation.cc:107] Waiting for socket to connect
0924 13:50:11.422282 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0924 13:50:11.422335 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:50:11.422550 (+ 215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:50:11.422553 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:50:11.422556 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 13:50:11.422628 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:50:11.422631 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:50:11.423939 (+ 1308us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:50:11.423940 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:50:11.424986 (+ 1046us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:50:11.424989 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:50:11.425628 (+ 639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:50:11.425630 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:50:11.425719 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:50:11.425733 (+ 14us) client_negotiation.cc:769] Sending connection context
0924 13:50:11.425796 (+ 63us) client_negotiation.cc:240] Negotiation successful
0924 13:50:11.425850 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":366,"thread_start_us":84,"threads_started":1}
I20260924 13:50:11.426159 28236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:50:11.421872 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.194:45113 (local address 127.0.0.1:42354)
0924 13:50:11.422201 (+ 329us) negotiation.cc:107] Waiting for socket to connect
0924 13:50:11.422250 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0924 13:50:11.422325 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:50:11.422621 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:50:11.422622 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:50:11.422624 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 13:50:11.422689 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:50:11.422691 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:50:11.423749 (+ 1058us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:50:11.423750 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:50:11.424693 (+ 943us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:50:11.424695 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:50:11.425742 (+ 1047us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:50:11.425743 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:50:11.425806 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:50:11.425811 (+ 5us) client_negotiation.cc:769] Sending connection context
0924 13:50:11.425861 (+ 50us) client_negotiation.cc:240] Negotiation successful
0924 13:50:11.425906 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":297,"thread_start_us":52,"threads_started":1}
I20260924 13:50:11.426152 28239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:50:11.422010 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42354 (local address 127.13.149.194:45113)
0924 13:50:11.422478 (+ 468us) server_negotiation.cc:206] Beginning negotiation
0924 13:50:11.422483 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 13:50:11.422492 (+ 9us) server_negotiation.cc:407] Connection header received
0924 13:50:11.422523 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:50:11.422525 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:50:11.422538 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:50:11.422615 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 13:50:11.422781 (+ 166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:50:11.423654 (+ 873us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:50:11.424818 (+ 1164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:50:11.425588 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:50:11.425654 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:50:11.425708 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0924 13:50:11.426003 (+ 295us) server_negotiation.cc:299] Negotiation successful
0924 13:50:11.426047 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":430,"thread_start_us":87,"threads_started":1}
I20260924 13:50:11.426666 28237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:50:11.422162 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.193:43429 (local address 127.0.0.1:46926)
0924 13:50:11.422720 (+ 558us) negotiation.cc:107] Waiting for socket to connect
0924 13:50:11.422731 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0924 13:50:11.422778 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:50:11.423201 (+ 423us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:50:11.423202 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:50:11.423204 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 13:50:11.423243 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:50:11.423245 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:50:11.424483 (+ 1238us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:50:11.424484 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:50:11.425551 (+ 1067us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:50:11.425554 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:50:11.426442 (+ 888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:50:11.426443 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:50:11.426503 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:50:11.426507 (+ 4us) client_negotiation.cc:769] Sending connection context
0924 13:50:11.426555 (+ 48us) client_negotiation.cc:240] Negotiation successful
0924 13:50:11.426603 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":538,"thread_start_us":61,"threads_started":1}
I20260924 13:50:11.426703 28241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:50:11.422246 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46926 (local address 127.13.149.193:43429)
0924 13:50:11.422420 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0924 13:50:11.422424 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 13:50:11.422962 (+ 538us) server_negotiation.cc:407] Connection header received
0924 13:50:11.423092 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:50:11.423095 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:50:11.423112 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:50:11.423191 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 13:50:11.423313 (+ 122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:50:11.424363 (+ 1050us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:50:11.425678 (+ 1315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:50:11.426362 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:50:11.426432 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:50:11.426473 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0924 13:50:11.426574 (+ 101us) server_negotiation.cc:299] Negotiation successful
0924 13:50:11.426618 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":147,"thread_start_us":73,"threads_started":1}
I20260924 13:50:11.427119 28159 tablet_service.cc:1467] Processing CreateTablet for tablet d5fb28bca78f4380bb28fa33322a35de (DEFAULT_TABLE table=test-table [id=19e3c3ba2a604baaac2c9e758cb207e9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 13:50:11.427119 28025 tablet_service.cc:1467] Processing CreateTablet for tablet d5fb28bca78f4380bb28fa33322a35de (DEFAULT_TABLE table=test-table [id=19e3c3ba2a604baaac2c9e758cb207e9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 13:50:11.427491 28025 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d5fb28bca78f4380bb28fa33322a35de. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 13:50:11.427491 28159 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d5fb28bca78f4380bb28fa33322a35de. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 13:50:11.427564 27891 tablet_service.cc:1467] Processing CreateTablet for tablet d5fb28bca78f4380bb28fa33322a35de (DEFAULT_TABLE table=test-table [id=19e3c3ba2a604baaac2c9e758cb207e9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 13:50:11.427809 27891 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d5fb28bca78f4380bb28fa33322a35de. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 13:50:11.430367 28242 tablet_bootstrap.cc:492] T d5fb28bca78f4380bb28fa33322a35de P 46b4fd024b654a5497f77167a74589cf: Bootstrap starting.
I20260924 13:50:11.430781 28243 tablet_bootstrap.cc:492] T d5fb28bca78f4380bb28fa33322a35de P 8423e2366dc04ab8b01dc9d932e8ee75: Bootstrap starting.
I20260924 13:50:11.430975 28242 tablet_bootstrap.cc:654] T d5fb28bca78f4380bb28fa33322a35de P 46b4fd024b654a5497f77167a74589cf: Neither blocks nor log segments found. Creating new log.
I20260924 13:50:11.431458 28243 tablet_bootstrap.cc:654] T d5fb28bca78f4380bb28fa33322a35de P 8423e2366dc04ab8b01dc9d932e8ee75: Neither blocks nor log segments found. Creating new log.
I20260924 13:50:11.431556 28242 log.cc:824] T d5fb28bca78f4380bb28fa33322a35de P 46b4fd024b654a5497f77167a74589cf: Log is configured to *not* fsync() on all Append() calls
I20260924 13:50:11.431959 28243 log.cc:824] T d5fb28bca78f4380bb28fa33322a35de P 8423e2366dc04ab8b01dc9d932e8ee75: Log is configured to *not* fsync() on all Append() calls
I20260924 13:50:11.432271 28244 tablet_bootstrap.cc:492] T d5fb28bca78f4380bb28fa33322a35de P 666006d5e083435b9d59291a298b266c: Bootstrap starting.
I20260924 13:50:11.432312 28242 tablet_bootstrap.cc:492] T d5fb28bca78f4380bb28fa33322a35de P 46b4fd024b654a5497f77167a74589cf: No bootstrap required, opened a new log
I20260924 13:50:11.432364 28242 ts_tablet_manager.cc:1397] T d5fb28bca78f4380bb28fa33322a35de P 46b4fd024b654a5497f77167a74589cf: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260924 13:50:11.432607 28243 tablet_bootstrap.cc:492] T d5fb28bca78f4380bb28fa33322a35de P 8423e2366dc04ab8b01dc9d932e8ee75: No bootstrap required, opened a new log
I20260924 13:50:11.432667 28243 ts_tablet_manager.cc:1397] T d5fb28bca78f4380bb28fa33322a35de P 8423e2366dc04ab8b01dc9d932e8ee75: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260924 13:50:11.432941 28244 tablet_bootstrap.cc:654] T d5fb28bca78f4380bb28fa33322a35de P 666006d5e083435b9d59291a298b266c: Neither blocks nor log segments found. Creating new log.
I20260924 13:50:11.433501 28244 log.cc:824] T d5fb28bca78f4380bb28fa33322a35de P 666006d5e083435b9d59291a298b266c: Log is configured to *not* fsync() on all Append() calls
I20260924 13:50:11.434276 28244 tablet_bootstrap.cc:492] T d5fb28bca78f4380bb28fa33322a35de P 666006d5e083435b9d59291a298b266c: No bootstrap required, opened a new log
I20260924 13:50:11.434345 28244 ts_tablet_manager.cc:1397] T d5fb28bca78f4380bb28fa33322a35de P 666006d5e083435b9d59291a298b266c: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260924 13:50:11.434355 28242 raft_consensus.cc:348] T d5fb28bca78f4380bb28fa33322a35de P 46b4fd024b654a5497f77167a74589cf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8423e2366dc04ab8b01dc9d932e8ee75" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 45095 } } peers { permanent_uuid: "46b4fd024b654a5497f77167a74589cf" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 43429 } } peers { permanent_uuid: "666006d5e083435b9d59291a298b266c" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 45113 } }
I20260924 13:50:11.434373 28243 raft_consensus.cc:348] T d5fb28bca78f4380bb28fa33322a35de P 8423e2366dc04ab8b01dc9d932e8ee75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8423e2366dc04ab8b01dc9d932e8ee75" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 45095 } } peers { permanent_uuid: "46b4fd024b654a5497f77167a74589cf" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 43429 } } peers { permanent_uuid: "666006d5e083435b9d59291a298b266c" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 45113 } }
I20260924 13:50:11.434517 28243 raft_consensus.cc:374] T d5fb28bca78f4380bb28fa33322a35de P 8423e2366dc04ab8b01dc9d932e8ee75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 13:50:11.434518 28242 raft_consensus.cc:374] T d5fb28bca78f4380bb28fa33322a35de P 46b4fd024b654a5497f77167a74589cf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 13:50:11.434549 28243 raft_consensus.cc:729] T d5fb28bca78f4380bb28fa33322a35de P 8423e2366dc04ab8b01dc9d932e8ee75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8423e2366dc04ab8b01dc9d932e8ee75, State: Initialized, Role: FOLLOWER
I20260924 13:50:11.434552 28242 raft_consensus.cc:729] T d5fb28bca78f4380bb28fa33322a35de P 46b4fd024b654a5497f77167a74589cf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 46b4fd024b654a5497f77167a74589cf, State: Initialized, Role: FOLLOWER
I20260924 13:50:11.434666 28242 consensus_queue.cc:260] T d5fb28bca78f4380bb28fa33322a35de P 46b4fd024b654a5497f77167a74589cf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8423e2366dc04ab8b01dc9d932e8ee75" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 45095 } } peers { permanent_uuid: "46b4fd024b654a5497f77167a74589cf" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 43429 } } peers { permanent_uuid: "666006d5e083435b9d59291a298b266c" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 45113 } }
I20260924 13:50:11.434724 28243 consensus_queue.cc:260] T d5fb28bca78f4380bb28fa33322a35de P 8423e2366dc04ab8b01dc9d932e8ee75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8423e2366dc04ab8b01dc9d932e8ee75" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 45095 } } peers { permanent_uuid: "46b4fd024b654a5497f77167a74589cf" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 43429 } } peers { permanent_uuid: "666006d5e083435b9d59291a298b266c" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 45113 } }
I20260924 13:50:11.434921 28242 ts_tablet_manager.cc:1428] T d5fb28bca78f4380bb28fa33322a35de P 46b4fd024b654a5497f77167a74589cf: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260924 13:50:11.435029 28225 heartbeater.cc:503] Master 127.13.149.254:36075 was elected leader, sending a full tablet report...
I20260924 13:50:11.435083 27957 heartbeater.cc:503] Master 127.13.149.254:36075 was elected leader, sending a full tablet report...
I20260924 13:50:11.435286 28243 ts_tablet_manager.cc:1428] T d5fb28bca78f4380bb28fa33322a35de P 8423e2366dc04ab8b01dc9d932e8ee75: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260924 13:50:11.436234 28244 raft_consensus.cc:348] T d5fb28bca78f4380bb28fa33322a35de P 666006d5e083435b9d59291a298b266c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8423e2366dc04ab8b01dc9d932e8ee75" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 45095 } } peers { permanent_uuid: "46b4fd024b654a5497f77167a74589cf" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 43429 } } peers { permanent_uuid: "666006d5e083435b9d59291a298b266c" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 45113 } }
I20260924 13:50:11.436363 28244 raft_consensus.cc:374] T d5fb28bca78f4380bb28fa33322a35de P 666006d5e083435b9d59291a298b266c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 13:50:11.436383 28244 raft_consensus.cc:729] T d5fb28bca78f4380bb28fa33322a35de P 666006d5e083435b9d59291a298b266c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 666006d5e083435b9d59291a298b266c, State: Initialized, Role: FOLLOWER
I20260924 13:50:11.436484 28244 consensus_queue.cc:260] T d5fb28bca78f4380bb28fa33322a35de P 666006d5e083435b9d59291a298b266c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8423e2366dc04ab8b01dc9d932e8ee75" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 45095 } } peers { permanent_uuid: "46b4fd024b654a5497f77167a74589cf" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 43429 } } peers { permanent_uuid: "666006d5e083435b9d59291a298b266c" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 45113 } }
I20260924 13:50:11.436718 28244 ts_tablet_manager.cc:1428] T d5fb28bca78f4380bb28fa33322a35de P 666006d5e083435b9d59291a298b266c: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260924 13:50:11.436756 28091 heartbeater.cc:503] Master 127.13.149.254:36075 was elected leader, sending a full tablet report...
W20260924 13:50:11.470376 28092 tablet.cc:2369] T d5fb28bca78f4380bb28fa33322a35de P 666006d5e083435b9d59291a298b266c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260924 13:50:11.491881 28250 raft_consensus.cc:482] T d5fb28bca78f4380bb28fa33322a35de P 666006d5e083435b9d59291a298b266c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 13:50:11.492038 28250 raft_consensus.cc:504] T d5fb28bca78f4380bb28fa33322a35de P 666006d5e083435b9d59291a298b266c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8423e2366dc04ab8b01dc9d932e8ee75" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 45095 } } peers { permanent_uuid: "46b4fd024b654a5497f77167a74589cf" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 43429 } } peers { permanent_uuid: "666006d5e083435b9d59291a298b266c" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 45113 } }
I20260924 13:50:11.492404 28250 leader_election.cc:288] T d5fb28bca78f4380bb28fa33322a35de P 666006d5e083435b9d59291a298b266c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8423e2366dc04ab8b01dc9d932e8ee75 (127.13.149.195:45095), 46b4fd024b654a5497f77167a74589cf (127.13.149.193:43429)
I20260924 13:50:11.496392 28238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:50:11.492638 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.149.194:33623 (local address 127.13.149.195:45095)
0924 13:50:11.492691 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0924 13:50:11.492694 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 13:50:11.492708 (+ 14us) server_negotiation.cc:407] Connection header received
0924 13:50:11.492802 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:50:11.492804 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:50:11.492813 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:50:11.492869 (+ 56us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 13:50:11.493058 (+ 189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:50:11.493822 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:50:11.495375 (+ 1553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:50:11.495984 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:50:11.496057 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:50:11.496066 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0924 13:50:11.496275 (+ 209us) server_negotiation.cc:299] Negotiation successful
0924 13:50:11.496322 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260924 13:50:11.496398 28093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:50:11.492563 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.195:45095 (local address 127.13.149.194:33623)
0924 13:50:11.492625 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0924 13:50:11.492638 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0924 13:50:11.492706 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:50:11.492888 (+ 182us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:50:11.492889 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:50:11.492891 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 13:50:11.492989 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:50:11.492991 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:50:11.493969 (+ 978us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:50:11.493970 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:50:11.495280 (+ 1310us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:50:11.495284 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:50:11.496063 (+ 779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:50:11.496064 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:50:11.496193 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:50:11.496198 (+ 5us) client_negotiation.cc:769] Sending connection context
0924 13:50:11.496262 (+ 64us) client_negotiation.cc:240] Negotiation successful
0924 13:50:11.496309 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40,"mutex_wait_us":64}
I20260924 13:50:11.496784 28241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:50:11.492769 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.149.194:49607 (local address 127.13.149.193:43429)
0924 13:50:11.492819 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0924 13:50:11.492821 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:50:11.492890 (+ 69us) server_negotiation.cc:407] Connection header received
0924 13:50:11.493006 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:50:11.493007 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:50:11.493014 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:50:11.493066 (+ 52us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 13:50:11.493178 (+ 112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:50:11.493879 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:50:11.495380 (+ 1501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:50:11.495983 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:50:11.496057 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:50:11.496066 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0924 13:50:11.496669 (+ 603us) server_negotiation.cc:299] Negotiation successful
0924 13:50:11.496720 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260924 13:50:11.497007 28251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:50:11.492668 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.193:43429 (local address 127.13.149.194:49607)
0924 13:50:11.492820 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0924 13:50:11.492828 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0924 13:50:11.492941 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:50:11.493077 (+ 136us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:50:11.493078 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:50:11.493079 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 13:50:11.493128 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:50:11.493129 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:50:11.493969 (+ 840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:50:11.493970 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:50:11.495281 (+ 1311us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:50:11.495284 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:50:11.496506 (+ 1222us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:50:11.496507 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:50:11.496576 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:50:11.496580 (+ 4us) client_negotiation.cc:769] Sending connection context
0924 13:50:11.496867 (+ 287us) client_negotiation.cc:240] Negotiation successful
0924 13:50:11.496940 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"mutex_wait_us":38,"thread_start_us":108,"threads_started":1}
I20260924 13:50:11.496944 28179 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d5fb28bca78f4380bb28fa33322a35de" candidate_uuid: "666006d5e083435b9d59291a298b266c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8423e2366dc04ab8b01dc9d932e8ee75" is_pre_election: true
I20260924 13:50:11.497120 28179 raft_consensus.cc:2454] T d5fb28bca78f4380bb28fa33322a35de P 8423e2366dc04ab8b01dc9d932e8ee75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 666006d5e083435b9d59291a298b266c in term 0.
I20260924 13:50:11.497351 27981 leader_election.cc:302] T d5fb28bca78f4380bb28fa33322a35de P 666006d5e083435b9d59291a298b266c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 666006d5e083435b9d59291a298b266c, 8423e2366dc04ab8b01dc9d932e8ee75; no voters:
I20260924 13:50:11.497416 27911 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d5fb28bca78f4380bb28fa33322a35de" candidate_uuid: "666006d5e083435b9d59291a298b266c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "46b4fd024b654a5497f77167a74589cf" is_pre_election: true
I20260924 13:50:11.497565 27911 raft_consensus.cc:2454] T d5fb28bca78f4380bb28fa33322a35de P 46b4fd024b654a5497f77167a74589cf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 666006d5e083435b9d59291a298b266c in term 0.
I20260924 13:50:11.497551 28250 raft_consensus.cc:2781] T d5fb28bca78f4380bb28fa33322a35de P 666006d5e083435b9d59291a298b266c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 13:50:11.497611 28250 raft_consensus.cc:482] T d5fb28bca78f4380bb28fa33322a35de P 666006d5e083435b9d59291a298b266c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 13:50:11.497643 28250 raft_consensus.cc:3037] T d5fb28bca78f4380bb28fa33322a35de P 666006d5e083435b9d59291a298b266c [term 0 FOLLOWER]: Advancing to term 1
I20260924 13:50:11.498400 28250 raft_consensus.cc:504] T d5fb28bca78f4380bb28fa33322a35de P 666006d5e083435b9d59291a298b266c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8423e2366dc04ab8b01dc9d932e8ee75" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 45095 } } peers { permanent_uuid: "46b4fd024b654a5497f77167a74589cf" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 43429 } } peers { permanent_uuid: "666006d5e083435b9d59291a298b266c" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 45113 } }
I20260924 13:50:11.498523 28250 leader_election.cc:288] T d5fb28bca78f4380bb28fa33322a35de P 666006d5e083435b9d59291a298b266c [CANDIDATE]: Term 1 election: Requested vote from peers 8423e2366dc04ab8b01dc9d932e8ee75 (127.13.149.195:45095), 46b4fd024b654a5497f77167a74589cf (127.13.149.193:43429)
I20260924 13:50:11.498694 27911 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d5fb28bca78f4380bb28fa33322a35de" candidate_uuid: "666006d5e083435b9d59291a298b266c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "46b4fd024b654a5497f77167a74589cf"
I20260924 13:50:11.498714 28179 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d5fb28bca78f4380bb28fa33322a35de" candidate_uuid: "666006d5e083435b9d59291a298b266c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8423e2366dc04ab8b01dc9d932e8ee75"
I20260924 13:50:11.498772 27911 raft_consensus.cc:3037] T d5fb28bca78f4380bb28fa33322a35de P 46b4fd024b654a5497f77167a74589cf [term 0 FOLLOWER]: Advancing to term 1
I20260924 13:50:11.498778 28179 raft_consensus.cc:3037] T d5fb28bca78f4380bb28fa33322a35de P 8423e2366dc04ab8b01dc9d932e8ee75 [term 0 FOLLOWER]: Advancing to term 1
I20260924 13:50:11.499477 27911 raft_consensus.cc:2454] T d5fb28bca78f4380bb28fa33322a35de P 46b4fd024b654a5497f77167a74589cf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 666006d5e083435b9d59291a298b266c in term 1.
I20260924 13:50:11.499476 28179 raft_consensus.cc:2454] T d5fb28bca78f4380bb28fa33322a35de P 8423e2366dc04ab8b01dc9d932e8ee75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 666006d5e083435b9d59291a298b266c in term 1.
I20260924 13:50:11.499634 27981 leader_election.cc:302] T d5fb28bca78f4380bb28fa33322a35de P 666006d5e083435b9d59291a298b266c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 666006d5e083435b9d59291a298b266c, 8423e2366dc04ab8b01dc9d932e8ee75; no voters:
I20260924 13:50:11.499728 28250 raft_consensus.cc:2781] T d5fb28bca78f4380bb28fa33322a35de P 666006d5e083435b9d59291a298b266c [term 1 FOLLOWER]: Leader election won for term 1
I20260924 13:50:11.499900 28250 raft_consensus.cc:686] T d5fb28bca78f4380bb28fa33322a35de P 666006d5e083435b9d59291a298b266c [term 1 LEADER]: Becoming Leader. State: Replica: 666006d5e083435b9d59291a298b266c, State: Running, Role: LEADER
I20260924 13:50:11.499992 28250 consensus_queue.cc:237] T d5fb28bca78f4380bb28fa33322a35de P 666006d5e083435b9d59291a298b266c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8423e2366dc04ab8b01dc9d932e8ee75" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 45095 } } peers { permanent_uuid: "46b4fd024b654a5497f77167a74589cf" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 43429 } } peers { permanent_uuid: "666006d5e083435b9d59291a298b266c" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 45113 } }
I20260924 13:50:11.500815 27765 catalog_manager.cc:5495] T d5fb28bca78f4380bb28fa33322a35de P 666006d5e083435b9d59291a298b266c reported cstate change: term changed from 0 to 1, leader changed from <none> to 666006d5e083435b9d59291a298b266c (127.13.149.194). New cstate: current_term: 1 leader_uuid: "666006d5e083435b9d59291a298b266c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8423e2366dc04ab8b01dc9d932e8ee75" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 45095 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "46b4fd024b654a5497f77167a74589cf" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 43429 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "666006d5e083435b9d59291a298b266c" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 45113 } health_report { overall_health: HEALTHY } } }
I20260924 13:50:11.517755 28239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:50:11.515226 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42366 (local address 127.13.149.194:45113)
0924 13:50:11.515389 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0924 13:50:11.515391 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:50:11.515401 (+ 10us) server_negotiation.cc:407] Connection header received
0924 13:50:11.515439 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:50:11.515439 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:50:11.515459 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:50:11.515538 (+ 79us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 13:50:11.515638 (+ 100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:50:11.516141 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:50:11.516897 (+ 756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:50:11.517159 (+ 262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:50:11.517216 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:50:11.517382 (+ 166us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 13:50:11.517487 (+ 105us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 13:50:11.517551 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0924 13:50:11.517622 (+ 71us) server_negotiation.cc:299] Negotiation successful
0924 13:50:11.517639 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":133}
I20260924 13:50:11.519589 28179 raft_consensus.cc:1261] T d5fb28bca78f4380bb28fa33322a35de P 8423e2366dc04ab8b01dc9d932e8ee75 [term 1 FOLLOWER]: Refusing update from remote peer 666006d5e083435b9d59291a298b266c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 13:50:11.519587 27911 raft_consensus.cc:1261] T d5fb28bca78f4380bb28fa33322a35de P 46b4fd024b654a5497f77167a74589cf [term 1 FOLLOWER]: Refusing update from remote peer 666006d5e083435b9d59291a298b266c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 13:50:11.519944 28252 consensus_queue.cc:1035] T d5fb28bca78f4380bb28fa33322a35de P 666006d5e083435b9d59291a298b266c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8423e2366dc04ab8b01dc9d932e8ee75" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 13:50:11.520068 28250 consensus_queue.cc:1035] T d5fb28bca78f4380bb28fa33322a35de P 666006d5e083435b9d59291a298b266c [LEADER]: Connected to new peer: Peer: permanent_uuid: "46b4fd024b654a5497f77167a74589cf" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 43429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 13:50:11.522541 28255 mvcc.cc:205] Tried to move back new op lower bound from 7332895995981631488 to 7332895995904507904. Current Snapshot: MvccSnapshot[applied={T|T < 7332895995981631488}]
I20260924 13:50:11.523378 28256 mvcc.cc:205] Tried to move back new op lower bound from 7332895995981631488 to 7332895995904507904. Current Snapshot: MvccSnapshot[applied={T|T < 7332895995981631488}]
I20260924 13:50:11.524559 27765 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48708:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 13:50:11.524658 27765 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48708:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 13:50:11.525396 27765 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ffaff00f2fee4d03bb1f71c504fabd7f: Sending DeleteTablet for 3 replicas of tablet d5fb28bca78f4380bb28fa33322a35de
I20260924 13:50:11.525775 13911 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJN9WRF/build/release/bin/kudu with pid 27828
I20260924 13:50:11.525756 28159 tablet_service.cc:1514] Processing DeleteTablet for tablet d5fb28bca78f4380bb28fa33322a35de with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 13:50:11 UTC) from {username='kudu', principal='kudu/127.13.149.254@KRBTEST.COM'} at 127.0.0.1:54614
I20260924 13:50:11.525786 27891 tablet_service.cc:1514] Processing DeleteTablet for tablet d5fb28bca78f4380bb28fa33322a35de with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 13:50:11 UTC) from {username='kudu', principal='kudu/127.13.149.254@KRBTEST.COM'} at 127.0.0.1:46926
I20260924 13:50:11.525947 28025 tablet_service.cc:1514] Processing DeleteTablet for tablet d5fb28bca78f4380bb28fa33322a35de with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 13:50:11 UTC) from {username='kudu', principal='kudu/127.13.149.254@KRBTEST.COM'} at 127.0.0.1:42354
I20260924 13:50:11.526412 28265 tablet_replica.cc:330] T d5fb28bca78f4380bb28fa33322a35de P 8423e2366dc04ab8b01dc9d932e8ee75: stopping tablet replica
I20260924 13:50:11.526499 28265 raft_consensus.cc:2229] T d5fb28bca78f4380bb28fa33322a35de P 8423e2366dc04ab8b01dc9d932e8ee75 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 13:50:11.526590 28265 raft_consensus.cc:2258] T d5fb28bca78f4380bb28fa33322a35de P 8423e2366dc04ab8b01dc9d932e8ee75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 13:50:11.527029 28265 ts_tablet_manager.cc:1905] T d5fb28bca78f4380bb28fa33322a35de P 8423e2366dc04ab8b01dc9d932e8ee75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 13:50:11.529080 28265 ts_tablet_manager.cc:1918] T d5fb28bca78f4380bb28fa33322a35de P 8423e2366dc04ab8b01dc9d932e8ee75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 13:50:11.529158 28265 log.cc:1196] T d5fb28bca78f4380bb28fa33322a35de P 8423e2366dc04ab8b01dc9d932e8ee75: Deleting WAL directory at /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790257629041645-13911-0/minicluster-data/ts-2/wal/wals/d5fb28bca78f4380bb28fa33322a35de
I20260924 13:50:11.529438 28265 ts_tablet_manager.cc:1939] T d5fb28bca78f4380bb28fa33322a35de P 8423e2366dc04ab8b01dc9d932e8ee75: Deleting consensus metadata
I20260924 13:50:11.529856 27753 catalog_manager.cc:4841] TS 8423e2366dc04ab8b01dc9d932e8ee75 (127.13.149.195:45095): tablet d5fb28bca78f4380bb28fa33322a35de (table test-table [id=19e3c3ba2a604baaac2c9e758cb207e9]) successfully deleted
W20260924 13:50:11.530884 27753 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.13.149.193:43429 (error 108)
I20260924 13:50:11.530933 13911 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJN9WRF/build/release/bin/kudu with pid 27962
I20260924 13:50:11.534772 28237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:50:11.534652 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.193:43429 (local address 127.0.0.1:46942)
0924 13:50:11.534699 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0924 13:50:11.534734 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.149.193:43429: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260924 13:50:11.534922 27753 catalog_manager.cc:4584] TS 46b4fd024b654a5497f77167a74589cf (127.13.149.193:43429): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet d5fb28bca78f4380bb28fa33322a35de: Network error: Client connection negotiation failed: client connection to 127.13.149.193:43429: connect: Connection refused (error 111)
I20260924 13:50:11.535944 13911 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJN9WRF/build/release/bin/kudu with pid 28096
I20260924 13:50:11.536012 28237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:50:11.535899 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.194:45113 (local address 127.0.0.1:42378)
0924 13:50:11.535957 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0924 13:50:11.535987 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.149.194:45113: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260924 13:50:11.540323 13911 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJN9WRF/build/release/bin/kudu with pid 27734
I20260924 13:50:11.544822 13911 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJN9WRF/build/release/bin/kudu
/tmp/dist-test-taskJN9WRF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790257629041645-13911-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790257629041645-13911-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790257629041645-13911-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790257629041645-13911-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.149.254:36075
--webserver_interface=127.13.149.254
--webserver_port=44463
--keytab_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.254.keytab
--principal=kudu/127.13.149.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.149.212:45289
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.149.254:36075
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790257629041645-13911-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790257629041645-13911-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790257629041645-13911-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 13:50:11.628690 28268 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 13:50:11.628964 28268 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 13:50:11.629000 28268 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 13:50:11.630775 28268 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 13:50:11.630841 28268 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 13:50:11.630859 28268 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 13:50:11.630878 28268 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 13:50:11.630903 28268 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 13:50:11.633018 28268 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.149.212:45289
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790257629041645-13911-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790257629041645-13911-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.13.149.254:36075
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.254.keytab
--principal=kudu/127.13.149.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.149.254:36075
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790257629041645-13911-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.149.254
--webserver_port=44463
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28268
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790257629041645-13911-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 12:29:37 UTC on ccycloud.quasar-iqzbxn.root.comops.site
build id 4670106
I20260924 13:50:11.633289 28268 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 13:50:11.633502 28268 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 13:50:11.645020 28275 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 13:50:11.645035 28274 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 13:50:11.645020 28277 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 13:50:11.650465 28268 server_base.cc:1034] running on GCE node
I20260924 13:50:11.650858 28268 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 13:50:11.651201 28268 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 13:50:11.652424 28268 hybrid_clock.cc:648] HybridClock initialized: now 1790257811652407 us; error 29 us; skew 500 ppm
Sep 24 13:50:11 dist-test-slave-sjl2 krb5kdc[27721](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257811, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.149.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 13:50:11.654981 28268 init.cc:377] Logged in from keytab as kudu/127.13.149.254@KRBTEST.COM (short username kudu)
I20260924 13:50:11.655709 28268 webserver.cc:460] Webserver started at http://127.13.149.254:44463/ using document root <none> and password file <none>
I20260924 13:50:11.655937 28268 fs_manager.cc:359] Metadata directory not provided
I20260924 13:50:11.656000 28268 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 13:50:11.657513 28268 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.001s
I20260924 13:50:11.658212 28284 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 13:50:11.658377 28268 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 13:50:11.658470 28268 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790257629041645-13911-0/minicluster-data/master-0/data,/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790257629041645-13911-0/minicluster-data/master-0/wal
uuid: "ffaff00f2fee4d03bb1f71c504fabd7f"
format_stamp: "Formatted at 2026-09-24 13:50:10 on dist-test-slave-sjl2"
server_key: "b5ae1bf96c07654927cebe93e7a64d6e"
server_key_iv: "0efeda2723646de6a9a598a3753139cd"
server_key_version: "encryptionkey@0"
I20260924 13:50:11.658912 28268 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790257629041645-13911-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790257629041645-13911-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790257629041645-13911-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 13:50:11.668704 28268 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 13:50:11.670835 28268 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 13:50:11.670985 28268 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 13:50:11.675146 28268 rpc_server.cc:307] RPC server started. Bound to: 127.13.149.254:36075
I20260924 13:50:11.675204 28336 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.149.254:36075 every 8 connection(s)
I20260924 13:50:11.675638 28268 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790257629041645-13911-0/minicluster-data/master-0/data/info.pb
I20260924 13:50:11.677336 28337 sys_catalog.cc:263] Verifying existing consensus state
I20260924 13:50:11.678000 28337 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ffaff00f2fee4d03bb1f71c504fabd7f: Bootstrap starting.
I20260924 13:50:11.679551 13911 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJN9WRF/build/release/bin/kudu as pid 28268
I20260924 13:50:11.679857 13911 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJN9WRF/build/release/bin/kudu
/tmp/dist-test-taskJN9WRF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790257629041645-13911-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790257629041645-13911-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790257629041645-13911-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790257629041645-13911-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.149.193:43429
--local_ip_for_outbound_sockets=127.13.149.193
--tserver_master_addrs=127.13.149.254:36075
--webserver_port=43713
--webserver_interface=127.13.149.193
--keytab_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.193.keytab
--principal=kudu/127.13.149.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.149.212:45289
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790257629041645-13911-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790257629041645-13911-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790257629041645-13911-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260924 13:50:11.686753 28337 log.cc:824] T 00000000000000000000000000000000 P ffaff00f2fee4d03bb1f71c504fabd7f: Log is configured to *not* fsync() on all Append() calls
I20260924 13:50:11.688669 28337 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ffaff00f2fee4d03bb1f71c504fabd7f: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260924 13:50:11.689000 28337 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ffaff00f2fee4d03bb1f71c504fabd7f: Bootstrap complete.
I20260924 13:50:11.691130 28337 raft_consensus.cc:348] T 00000000000000000000000000000000 P ffaff00f2fee4d03bb1f71c504fabd7f [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ffaff00f2fee4d03bb1f71c504fabd7f" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 36075 } }
I20260924 13:50:11.691360 28337 raft_consensus.cc:729] T 00000000000000000000000000000000 P ffaff00f2fee4d03bb1f71c504fabd7f [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: ffaff00f2fee4d03bb1f71c504fabd7f, State: Initialized, Role: FOLLOWER
I20260924 13:50:11.691509 28337 consensus_queue.cc:260] T 00000000000000000000000000000000 P ffaff00f2fee4d03bb1f71c504fabd7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ffaff00f2fee4d03bb1f71c504fabd7f" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 36075 } }
I20260924 13:50:11.691582 28337 raft_consensus.cc:388] T 00000000000000000000000000000000 P ffaff00f2fee4d03bb1f71c504fabd7f [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 13:50:11.691637 28337 raft_consensus.cc:482] T 00000000000000000000000000000000 P ffaff00f2fee4d03bb1f71c504fabd7f [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 13:50:11.691685 28337 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ffaff00f2fee4d03bb1f71c504fabd7f [term 1 FOLLOWER]: Advancing to term 2
I20260924 13:50:11.692715 28337 raft_consensus.cc:504] T 00000000000000000000000000000000 P ffaff00f2fee4d03bb1f71c504fabd7f [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ffaff00f2fee4d03bb1f71c504fabd7f" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 36075 } }
I20260924 13:50:11.692809 28337 leader_election.cc:302] T 00000000000000000000000000000000 P ffaff00f2fee4d03bb1f71c504fabd7f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ffaff00f2fee4d03bb1f71c504fabd7f; no voters:
I20260924 13:50:11.692991 28337 leader_election.cc:288] T 00000000000000000000000000000000 P ffaff00f2fee4d03bb1f71c504fabd7f [CANDIDATE]: Term 2 election: Requested vote from peers
I20260924 13:50:11.693059 28341 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ffaff00f2fee4d03bb1f71c504fabd7f [term 2 FOLLOWER]: Leader election won for term 2
I20260924 13:50:11.693234 28341 raft_consensus.cc:686] T 00000000000000000000000000000000 P ffaff00f2fee4d03bb1f71c504fabd7f [term 2 LEADER]: Becoming Leader. State: Replica: ffaff00f2fee4d03bb1f71c504fabd7f, State: Running, Role: LEADER
I20260924 13:50:11.693266 28337 sys_catalog.cc:564] T 00000000000000000000000000000000 P ffaff00f2fee4d03bb1f71c504fabd7f [sys.catalog]: configured and running, proceeding with master startup.
I20260924 13:50:11.693348 28341 consensus_queue.cc:237] T 00000000000000000000000000000000 P ffaff00f2fee4d03bb1f71c504fabd7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ffaff00f2fee4d03bb1f71c504fabd7f" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 36075 } }
I20260924 13:50:11.693674 28341 sys_catalog.cc:455] T 00000000000000000000000000000000 P ffaff00f2fee4d03bb1f71c504fabd7f [sys.catalog]: SysCatalogTable state changed. Reason: New leader ffaff00f2fee4d03bb1f71c504fabd7f. Latest consensus state: current_term: 2 leader_uuid: "ffaff00f2fee4d03bb1f71c504fabd7f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ffaff00f2fee4d03bb1f71c504fabd7f" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 36075 } } }
I20260924 13:50:11.693747 28341 sys_catalog.cc:458] T 00000000000000000000000000000000 P ffaff00f2fee4d03bb1f71c504fabd7f [sys.catalog]: This master's current role is: LEADER
I20260924 13:50:11.693884 28342 sys_catalog.cc:455] T 00000000000000000000000000000000 P ffaff00f2fee4d03bb1f71c504fabd7f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "ffaff00f2fee4d03bb1f71c504fabd7f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ffaff00f2fee4d03bb1f71c504fabd7f" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 36075 } } }
I20260924 13:50:11.693943 28342 sys_catalog.cc:458] T 00000000000000000000000000000000 P ffaff00f2fee4d03bb1f71c504fabd7f [sys.catalog]: This master's current role is: LEADER
I20260924 13:50:11.694044 28349 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 13:50:11.694824 28349 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 13:50:11.694932 28349 catalog_manager.cc:1259] Loaded cluster ID: da0001853c6b4ac9b2a8cdec412b8ea4
I20260924 13:50:11.694969 28349 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 13:50:11.695382 28349 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260924 13:50:11.700385 28349 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260924 13:50:11.702986 28349 catalog_manager.cc:1504] Loading token signing keys...
I20260924 13:50:11.703198 28349 catalog_manager.cc:5879] T 00000000000000000000000000000000 P ffaff00f2fee4d03bb1f71c504fabd7f: Loaded TSK: 0
I20260924 13:50:11.703415 28349 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260924 13:50:11.768589 28339 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 13:50:11.768813 28339 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 13:50:11.768841 28339 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 13:50:11.770088 28339 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 13:50:11.770139 28339 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 13:50:11.770169 28339 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.149.193
I20260924 13:50:11.771672 28339 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.149.212:45289
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790257629041645-13911-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790257629041645-13911-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.193.keytab
--principal=kudu/127.13.149.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.149.193:43429
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790257629041645-13911-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.149.193
--webserver_port=43713
--tserver_master_addrs=127.13.149.254:36075
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28339
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.149.193
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790257629041645-13911-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 12:29:37 UTC on ccycloud.quasar-iqzbxn.root.comops.site
build id 4670106
I20260924 13:50:11.771920 28339 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 13:50:11.772166 28339 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 13:50:11.777369 28364 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 13:50:11.777390 28365 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 13:50:11.777496 28339 server_base.cc:1034] running on GCE node
W20260924 13:50:11.777606 28367 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 13:50:11.778003 28339 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 13:50:11.778316 28339 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 13:50:11.779480 28339 hybrid_clock.cc:648] HybridClock initialized: now 1790257811779466 us; error 35 us; skew 500 ppm
Sep 24 13:50:11 dist-test-slave-sjl2 krb5kdc[27721](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257811, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.149.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 13:50:11.782277 28339 init.cc:377] Logged in from keytab as kudu/127.13.149.193@KRBTEST.COM (short username kudu)
I20260924 13:50:11.782804 28339 webserver.cc:460] Webserver started at http://127.13.149.193:43713/ using document root <none> and password file <none>
I20260924 13:50:11.783047 28339 fs_manager.cc:359] Metadata directory not provided
I20260924 13:50:11.783113 28339 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 13:50:11.784572 28339 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260924 13:50:11.785267 28374 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 13:50:11.785451 28339 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260924 13:50:11.785537 28339 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790257629041645-13911-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790257629041645-13911-0/minicluster-data/ts-0/wal
uuid: "46b4fd024b654a5497f77167a74589cf"
format_stamp: "Formatted at 2026-09-24 13:50:11 on dist-test-slave-sjl2"
server_key: "421ffd5752f236310d95d01d0d561f72"
server_key_iv: "3279b4f8ab63fc4df9ce5e8ad6474835"
server_key_version: "encryptionkey@0"
I20260924 13:50:11.785920 28339 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790257629041645-13911-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790257629041645-13911-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790257629041645-13911-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 13:50:11.800184 28339 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 13:50:11.802521 28339 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 13:50:11.802675 28339 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 13:50:11.802942 28339 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 13:50:11.803434 28381 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260924 13:50:11.804881 28339 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260924 13:50:11.804970 28339 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260924 13:50:11.805007 28339 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260924 13:50:11.805651 28339 ts_tablet_manager.cc:610] Registered 1 tablets
I20260924 13:50:11.805696 28339 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260924 13:50:11.805760 28381 tablet_bootstrap.cc:492] T d5fb28bca78f4380bb28fa33322a35de P 46b4fd024b654a5497f77167a74589cf: Bootstrap starting.
I20260924 13:50:11.813268 28339 rpc_server.cc:307] RPC server started. Bound to: 127.13.149.193:43429
I20260924 13:50:11.813802 28339 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790257629041645-13911-0/minicluster-data/ts-0/data/info.pb
I20260924 13:50:11.814286 28491 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.149.193:43429 every 8 connection(s)
I20260924 13:50:11.817044 28381 log.cc:824] T d5fb28bca78f4380bb28fa33322a35de P 46b4fd024b654a5497f77167a74589cf: Log is configured to *not* fsync() on all Append() calls
I20260924 13:50:11.818259 13911 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJN9WRF/build/release/bin/kudu as pid 28339
I20260924 13:50:11.818516 13911 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJN9WRF/build/release/bin/kudu
/tmp/dist-test-taskJN9WRF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790257629041645-13911-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790257629041645-13911-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790257629041645-13911-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790257629041645-13911-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.149.194:45113
--local_ip_for_outbound_sockets=127.13.149.194
--tserver_master_addrs=127.13.149.254:36075
--webserver_port=42307
--webserver_interface=127.13.149.194
--keytab_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.194.keytab
--principal=kudu/127.13.149.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.149.212:45289
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790257629041645-13911-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790257629041645-13911-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790257629041645-13911-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260924 13:50:11.819736 28381 tablet_bootstrap.cc:492] T d5fb28bca78f4380bb28fa33322a35de P 46b4fd024b654a5497f77167a74589cf: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260924 13:50:11.820010 28381 tablet_bootstrap.cc:492] T d5fb28bca78f4380bb28fa33322a35de P 46b4fd024b654a5497f77167a74589cf: Bootstrap complete.
I20260924 13:50:11.821154 28381 ts_tablet_manager.cc:1397] T d5fb28bca78f4380bb28fa33322a35de P 46b4fd024b654a5497f77167a74589cf: Time spent bootstrapping tablet: real 0.015s user 0.000s sys 0.015s
Sep 24 13:50:11 dist-test-slave-sjl2 krb5kdc[27721](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257811, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.149.193@KRBTEST.COM for kudu/127.13.149.254@KRBTEST.COM
I20260924 13:50:11.822729 28381 raft_consensus.cc:348] T d5fb28bca78f4380bb28fa33322a35de P 46b4fd024b654a5497f77167a74589cf [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8423e2366dc04ab8b01dc9d932e8ee75" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 45095 } } peers { permanent_uuid: "46b4fd024b654a5497f77167a74589cf" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 43429 } } peers { permanent_uuid: "666006d5e083435b9d59291a298b266c" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 45113 } }
I20260924 13:50:11.823098 28381 raft_consensus.cc:729] T d5fb28bca78f4380bb28fa33322a35de P 46b4fd024b654a5497f77167a74589cf [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 46b4fd024b654a5497f77167a74589cf, State: Initialized, Role: FOLLOWER
I20260924 13:50:11.823282 28381 consensus_queue.cc:260] T d5fb28bca78f4380bb28fa33322a35de P 46b4fd024b654a5497f77167a74589cf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8423e2366dc04ab8b01dc9d932e8ee75" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 45095 } } peers { permanent_uuid: "46b4fd024b654a5497f77167a74589cf" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 43429 } } peers { permanent_uuid: "666006d5e083435b9d59291a298b266c" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 45113 } }
I20260924 13:50:11.823580 28381 ts_tablet_manager.cc:1428] T d5fb28bca78f4380bb28fa33322a35de P 46b4fd024b654a5497f77167a74589cf: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260924 13:50:11.827447 28494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:50:11.814550 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:36075 (local address 127.13.149.193:40257)
0924 13:50:11.815076 (+ 526us) negotiation.cc:107] Waiting for socket to connect
0924 13:50:11.815122 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0924 13:50:11.815728 (+ 606us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:50:11.816385 (+ 657us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:50:11.816391 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:50:11.816804 (+ 413us) client_negotiation.cc:189] Negotiated authn=SASL
0924 13:50:11.817124 (+ 320us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:50:11.817131 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:50:11.818656 (+ 1525us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:50:11.818657 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:50:11.819893 (+ 1236us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:50:11.819896 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:50:11.820805 (+ 909us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:50:11.820806 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:50:11.820960 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:50:11.821584 (+ 624us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 13:50:11.821602 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0924 13:50:11.823714 (+ 2112us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 13:50:11.825902 (+ 2188us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 13:50:11.825908 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 13:50:11.825909 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 13:50:11.826129 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 13:50:11.826415 (+ 286us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 13:50:11.826415 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 13:50:11.826416 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 13:50:11.826502 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 13:50:11.826947 (+ 445us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 13:50:11.826950 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 13:50:11.827061 (+ 111us) client_negotiation.cc:769] Sending connection context
0924 13:50:11.827137 (+ 76us) client_negotiation.cc:240] Negotiation successful
0924 13:50:11.827267 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":479,"thread_start_us":86,"threads_started":1}
I20260924 13:50:11.827579 28495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:50:11.814906 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.149.193:40257 (local address 127.13.149.254:36075)
0924 13:50:11.815919 (+ 1013us) server_negotiation.cc:206] Beginning negotiation
0924 13:50:11.815928 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0924 13:50:11.815966 (+ 38us) server_negotiation.cc:407] Connection header received
0924 13:50:11.816092 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:50:11.816098 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:50:11.816149 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:50:11.816278 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:50:11.817289 (+ 1011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:50:11.818521 (+ 1232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:50:11.819998 (+ 1477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:50:11.820581 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:50:11.820679 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:50:11.823839 (+ 3160us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:50:11.823844 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:50:11.823844 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 13:50:11.823858 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:50:11.825816 (+ 1958us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:50:11.826210 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:50:11.826214 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:50:11.826214 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:50:11.826352 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:50:11.826556 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:50:11.826557 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:50:11.826557 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:50:11.826881 (+ 324us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:50:11.826978 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0924 13:50:11.827219 (+ 241us) server_negotiation.cc:299] Negotiation successful
0924 13:50:11.827381 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":961,"thread_start_us":61,"threads_started":1}
I20260924 13:50:11.828222 28492 heartbeater.cc:347] Connected to a master server at 127.13.149.254:36075
I20260924 13:50:11.828346 28492 heartbeater.cc:464] Registering TS with master...
I20260924 13:50:11.828514 28492 heartbeater.cc:511] Master 127.13.149.254:36075 requested a full tablet report, sending...
I20260924 13:50:11.829346 28301 ts_manager.cc:194] Registered new tserver with Master: 46b4fd024b654a5497f77167a74589cf (127.13.149.193:43429)
I20260924 13:50:11.830627 28301 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.149.193@KRBTEST.COM'} at 127.13.149.193:40257
I20260924 13:50:11.831513 28492 heartbeater.cc:503] Master 127.13.149.254:36075 was elected leader, sending a full tablet report...
Sep 24 13:50:11 dist-test-slave-sjl2 krb5kdc[27721](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257811, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.149.254@KRBTEST.COM for kudu/127.13.149.193@KRBTEST.COM
I20260924 13:50:11.843652 28500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:50:11.830092 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46944 (local address 127.13.149.193:43429)
0924 13:50:11.830289 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0924 13:50:11.830293 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 13:50:11.830455 (+ 162us) server_negotiation.cc:407] Connection header received
0924 13:50:11.830542 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:50:11.830544 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:50:11.830574 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:50:11.830631 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:50:11.831179 (+ 548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:50:11.832309 (+ 1130us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:50:11.833243 (+ 934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:50:11.833723 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:50:11.833792 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:50:11.840491 (+ 6699us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:50:11.840504 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:50:11.840504 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 13:50:11.840513 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:50:11.842283 (+ 1770us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:50:11.842774 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:50:11.842777 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:50:11.842777 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:50:11.842845 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:50:11.843026 (+ 181us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:50:11.843026 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:50:11.843027 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:50:11.843249 (+ 222us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:50:11.843319 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0924 13:50:11.843493 (+ 174us) server_negotiation.cc:299] Negotiation successful
0924 13:50:11.843578 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":18,"server-negotiator.queue_time_us":168,"thread_start_us":91,"threads_started":1}
I20260924 13:50:11.843703 28499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:50:11.830020 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.193:43429 (local address 127.0.0.1:46944)
0924 13:50:11.830347 (+ 327us) negotiation.cc:107] Waiting for socket to connect
0924 13:50:11.830387 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0924 13:50:11.830456 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:50:11.830664 (+ 208us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:50:11.830669 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:50:11.831033 (+ 364us) client_negotiation.cc:189] Negotiated authn=SASL
0924 13:50:11.831101 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:50:11.831104 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:50:11.832417 (+ 1313us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:50:11.832418 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:50:11.833129 (+ 711us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:50:11.833133 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:50:11.833825 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:50:11.833826 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:50:11.833893 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:50:11.838684 (+ 4791us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 13:50:11.838691 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0924 13:50:11.840347 (+ 1656us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 13:50:11.842383 (+ 2036us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 13:50:11.842394 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 13:50:11.842396 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 13:50:11.842695 (+ 299us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 13:50:11.842900 (+ 205us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 13:50:11.842901 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 13:50:11.842901 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 13:50:11.842972 (+ 71us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 13:50:11.843310 (+ 338us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 13:50:11.843317 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 13:50:11.843380 (+ 63us) client_negotiation.cc:769] Sending connection context
0924 13:50:11.843457 (+ 77us) client_negotiation.cc:240] Negotiation successful
0924 13:50:11.843600 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":304,"thread_start_us":162,"threads_started":1}
I20260924 13:50:11.844238 28424 tablet_service.cc:1514] Processing DeleteTablet for tablet d5fb28bca78f4380bb28fa33322a35de with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 13:50:11 UTC) from {username='kudu', principal='kudu/127.13.149.254@KRBTEST.COM'} at 127.0.0.1:46944
I20260924 13:50:11.844798 28501 tablet_replica.cc:330] stopping tablet replica
I20260924 13:50:11.844950 28501 raft_consensus.cc:2229] T d5fb28bca78f4380bb28fa33322a35de P 46b4fd024b654a5497f77167a74589cf [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 13:50:11.845027 28501 raft_consensus.cc:2258] T d5fb28bca78f4380bb28fa33322a35de P 46b4fd024b654a5497f77167a74589cf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 13:50:11.845359 28501 ts_tablet_manager.cc:1905] T d5fb28bca78f4380bb28fa33322a35de P 46b4fd024b654a5497f77167a74589cf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 13:50:11.847728 28501 ts_tablet_manager.cc:1918] T d5fb28bca78f4380bb28fa33322a35de P 46b4fd024b654a5497f77167a74589cf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 13:50:11.847810 28501 log.cc:1196] T d5fb28bca78f4380bb28fa33322a35de P 46b4fd024b654a5497f77167a74589cf: Deleting WAL directory at /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790257629041645-13911-0/minicluster-data/ts-0/wal/wals/d5fb28bca78f4380bb28fa33322a35de
I20260924 13:50:11.848012 28501 ts_tablet_manager.cc:1939] T d5fb28bca78f4380bb28fa33322a35de P 46b4fd024b654a5497f77167a74589cf: Deleting consensus metadata
I20260924 13:50:11.848691 28288 catalog_manager.cc:4841] TS 46b4fd024b654a5497f77167a74589cf (127.13.149.193:43429): tablet d5fb28bca78f4380bb28fa33322a35de (table test-table [id=19e3c3ba2a604baaac2c9e758cb207e9]) successfully deleted
W20260924 13:50:11.906834 28497 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 13:50:11.907075 28497 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 13:50:11.907101 28497 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 13:50:11.908391 28497 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 13:50:11.908449 28497 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 13:50:11.908479 28497 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.149.194
I20260924 13:50:11.910125 28497 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.149.212:45289
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790257629041645-13911-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790257629041645-13911-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.194.keytab
--principal=kudu/127.13.149.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.149.194:45113
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790257629041645-13911-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.149.194
--webserver_port=42307
--tserver_master_addrs=127.13.149.254:36075
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28497
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.149.194
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790257629041645-13911-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 12:29:37 UTC on ccycloud.quasar-iqzbxn.root.comops.site
build id 4670106
I20260924 13:50:11.910386 28497 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 13:50:11.910614 28497 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 13:50:11.915869 28507 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 13:50:11.915869 28509 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 13:50:11.916000 28497 server_base.cc:1034] running on GCE node
W20260924 13:50:11.915889 28506 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 13:50:11.916291 28497 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 13:50:11.916498 28497 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 13:50:11.917634 28497 hybrid_clock.cc:648] HybridClock initialized: now 1790257811917623 us; error 27 us; skew 500 ppm
Sep 24 13:50:11 dist-test-slave-sjl2 krb5kdc[27721](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257811, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.149.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 13:50:11.920045 28497 init.cc:377] Logged in from keytab as kudu/127.13.149.194@KRBTEST.COM (short username kudu)
I20260924 13:50:11.920444 28497 webserver.cc:460] Webserver started at http://127.13.149.194:42307/ using document root <none> and password file <none>
I20260924 13:50:11.920630 28497 fs_manager.cc:359] Metadata directory not provided
I20260924 13:50:11.920678 28497 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 13:50:11.922096 28497 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260924 13:50:11.922704 28516 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 13:50:11.922879 28497 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 13:50:11.922961 28497 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790257629041645-13911-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790257629041645-13911-0/minicluster-data/ts-1/wal
uuid: "666006d5e083435b9d59291a298b266c"
format_stamp: "Formatted at 2026-09-24 13:50:11 on dist-test-slave-sjl2"
server_key: "44b1ebca15f1cd56048d3c315cd6857a"
server_key_iv: "a09f710677d3ee077cb6294c3dec6b22"
server_key_version: "encryptionkey@0"
I20260924 13:50:11.923324 28497 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790257629041645-13911-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790257629041645-13911-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790257629041645-13911-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 13:50:11.944892 28497 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 13:50:11.946859 28497 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 13:50:11.946974 28497 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 13:50:11.947163 28497 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 13:50:11.947536 28523 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260924 13:50:11.949028 28497 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260924 13:50:11.949082 28497 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260924 13:50:11.949110 28497 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260924 13:50:11.949747 28497 ts_tablet_manager.cc:610] Registered 1 tablets
I20260924 13:50:11.949777 28497 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260924 13:50:11.949867 28523 tablet_bootstrap.cc:492] T d5fb28bca78f4380bb28fa33322a35de P 666006d5e083435b9d59291a298b266c: Bootstrap starting.
I20260924 13:50:11.956808 28497 rpc_server.cc:307] RPC server started. Bound to: 127.13.149.194:45113
I20260924 13:50:11.956842 28630 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.149.194:45113 every 8 connection(s)
I20260924 13:50:11.957346 28497 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790257629041645-13911-0/minicluster-data/ts-1/data/info.pb
I20260924 13:50:11.959905 28523 log.cc:824] T d5fb28bca78f4380bb28fa33322a35de P 666006d5e083435b9d59291a298b266c: Log is configured to *not* fsync() on all Append() calls
I20260924 13:50:11.962366 28523 tablet_bootstrap.cc:492] T d5fb28bca78f4380bb28fa33322a35de P 666006d5e083435b9d59291a298b266c: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260924 13:50:11.962653 28523 tablet_bootstrap.cc:492] T d5fb28bca78f4380bb28fa33322a35de P 666006d5e083435b9d59291a298b266c: Bootstrap complete.
I20260924 13:50:11.963554 28523 ts_tablet_manager.cc:1397] T d5fb28bca78f4380bb28fa33322a35de P 666006d5e083435b9d59291a298b266c: Time spent bootstrapping tablet: real 0.014s user 0.003s sys 0.008s
Sep 24 13:50:11 dist-test-slave-sjl2 krb5kdc[27721](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257811, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.149.194@KRBTEST.COM for kudu/127.13.149.254@KRBTEST.COM
I20260924 13:50:11.964579 28523 raft_consensus.cc:348] T d5fb28bca78f4380bb28fa33322a35de P 666006d5e083435b9d59291a298b266c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8423e2366dc04ab8b01dc9d932e8ee75" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 45095 } } peers { permanent_uuid: "46b4fd024b654a5497f77167a74589cf" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 43429 } } peers { permanent_uuid: "666006d5e083435b9d59291a298b266c" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 45113 } }
I20260924 13:50:11.964874 28523 raft_consensus.cc:729] T d5fb28bca78f4380bb28fa33322a35de P 666006d5e083435b9d59291a298b266c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 666006d5e083435b9d59291a298b266c, State: Initialized, Role: FOLLOWER
I20260924 13:50:11.965057 13911 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJN9WRF/build/release/bin/kudu as pid 28497
I20260924 13:50:11.965042 28523 consensus_queue.cc:260] T d5fb28bca78f4380bb28fa33322a35de P 666006d5e083435b9d59291a298b266c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8423e2366dc04ab8b01dc9d932e8ee75" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 45095 } } peers { permanent_uuid: "46b4fd024b654a5497f77167a74589cf" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 43429 } } peers { permanent_uuid: "666006d5e083435b9d59291a298b266c" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 45113 } }
I20260924 13:50:11.965312 28523 ts_tablet_manager.cc:1428] T d5fb28bca78f4380bb28fa33322a35de P 666006d5e083435b9d59291a298b266c: Time spent starting tablet: real 0.002s user 0.004s sys 0.000s
I20260924 13:50:11.965338 13911 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJN9WRF/build/release/bin/kudu
/tmp/dist-test-taskJN9WRF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790257629041645-13911-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790257629041645-13911-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790257629041645-13911-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790257629041645-13911-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.149.195:45095
--local_ip_for_outbound_sockets=127.13.149.195
--tserver_master_addrs=127.13.149.254:36075
--webserver_port=41563
--webserver_interface=127.13.149.195
--keytab_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.195.keytab
--principal=kudu/127.13.149.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.149.212:45289
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790257629041645-13911-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790257629041645-13911-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790257629041645-13911-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260924 13:50:11.967141 28495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:50:11.958121 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.149.194:32907 (local address 127.13.149.254:36075)
0924 13:50:11.958178 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0924 13:50:11.958180 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:50:11.958356 (+ 176us) server_negotiation.cc:407] Connection header received
0924 13:50:11.958839 (+ 483us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:50:11.958841 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:50:11.958854 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:50:11.958903 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:50:11.959514 (+ 611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:50:11.960042 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:50:11.961403 (+ 1361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:50:11.961613 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:50:11.961671 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:50:11.964091 (+ 2420us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:50:11.964092 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:50:11.964093 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 13:50:11.964101 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:50:11.965709 (+ 1608us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:50:11.966078 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:50:11.966079 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:50:11.966079 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:50:11.966142 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:50:11.966411 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:50:11.966411 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:50:11.966411 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:50:11.966550 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:50:11.966620 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0924 13:50:11.966996 (+ 376us) server_negotiation.cc:299] Negotiation successful
0924 13:50:11.967076 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260924 13:50:11.967455 28633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:50:11.958052 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:36075 (local address 127.13.149.194:32907)
0924 13:50:11.958222 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0924 13:50:11.958265 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0924 13:50:11.958731 (+ 466us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:50:11.958961 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:50:11.958965 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:50:11.959223 (+ 258us) client_negotiation.cc:189] Negotiated authn=SASL
0924 13:50:11.959421 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:50:11.959426 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:50:11.960138 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:50:11.960139 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:50:11.961328 (+ 1189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:50:11.961331 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:50:11.961710 (+ 379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:50:11.961711 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:50:11.961863 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:50:11.962469 (+ 606us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 13:50:11.962478 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0924 13:50:11.963996 (+ 1518us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 13:50:11.965792 (+ 1796us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 13:50:11.965796 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 13:50:11.965798 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 13:50:11.966010 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 13:50:11.966231 (+ 221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 13:50:11.966232 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 13:50:11.966232 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 13:50:11.966348 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 13:50:11.966649 (+ 301us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 13:50:11.966659 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 13:50:11.966844 (+ 185us) client_negotiation.cc:769] Sending connection context
0924 13:50:11.966983 (+ 139us) client_negotiation.cc:240] Negotiation successful
0924 13:50:11.967239 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":80,"threads_started":1}
I20260924 13:50:11.967988 28631 heartbeater.cc:347] Connected to a master server at 127.13.149.254:36075
I20260924 13:50:11.968086 28631 heartbeater.cc:464] Registering TS with master...
I20260924 13:50:11.968266 28631 heartbeater.cc:511] Master 127.13.149.254:36075 requested a full tablet report, sending...
I20260924 13:50:11.968884 28301 ts_manager.cc:194] Registered new tserver with Master: 666006d5e083435b9d59291a298b266c (127.13.149.194:45113)
I20260924 13:50:11.969496 28301 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.149.194@KRBTEST.COM'} at 127.13.149.194:32907
I20260924 13:50:11.970227 28631 heartbeater.cc:503] Master 127.13.149.254:36075 was elected leader, sending a full tablet report...
Sep 24 13:50:11 dist-test-slave-sjl2 krb5kdc[27721](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257811, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.149.254@KRBTEST.COM for kudu/127.13.149.194@KRBTEST.COM
I20260924 13:50:11.977524 28499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:50:11.969260 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.194:45113 (local address 127.0.0.1:42388)
0924 13:50:11.969317 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0924 13:50:11.969329 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0924 13:50:11.969379 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:50:11.969751 (+ 372us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:50:11.969753 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:50:11.969925 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0924 13:50:11.969977 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:50:11.969979 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:50:11.971260 (+ 1281us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:50:11.971261 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:50:11.972047 (+ 786us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:50:11.972050 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:50:11.972504 (+ 454us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:50:11.972505 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:50:11.972578 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:50:11.973036 (+ 458us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 13:50:11.973041 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 13:50:11.974476 (+ 1435us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 13:50:11.976426 (+ 1950us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 13:50:11.976427 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 13:50:11.976427 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 13:50:11.976621 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 13:50:11.976829 (+ 208us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 13:50:11.976829 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 13:50:11.976830 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 13:50:11.976883 (+ 53us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 13:50:11.977232 (+ 349us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 13:50:11.977233 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 13:50:11.977282 (+ 49us) client_negotiation.cc:769] Sending connection context
0924 13:50:11.977351 (+ 69us) client_negotiation.cc:240] Negotiation successful
0924 13:50:11.977449 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36}
I20260924 13:50:11.977583 28637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:50:11.969390 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42388 (local address 127.13.149.194:45113)
0924 13:50:11.969555 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0924 13:50:11.969567 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0924 13:50:11.969577 (+ 10us) server_negotiation.cc:407] Connection header received
0924 13:50:11.969615 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:50:11.969617 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:50:11.969636 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:50:11.969743 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:50:11.970414 (+ 671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:50:11.971174 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:50:11.972137 (+ 963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:50:11.972430 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:50:11.972492 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:50:11.974559 (+ 2067us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:50:11.974570 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:50:11.974570 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 13:50:11.974579 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:50:11.976324 (+ 1745us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:50:11.976699 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:50:11.976702 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:50:11.976703 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:50:11.976771 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:50:11.976961 (+ 190us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:50:11.976961 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:50:11.976962 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:50:11.977144 (+ 182us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:50:11.977223 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0924 13:50:11.977389 (+ 166us) server_negotiation.cc:299] Negotiation successful
0924 13:50:11.977485 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":128,"thread_start_us":83,"threads_started":1}
I20260924 13:50:11.978001 28565 tablet_service.cc:1514] Processing DeleteTablet for tablet d5fb28bca78f4380bb28fa33322a35de with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 13:50:11 UTC) from {username='kudu', principal='kudu/127.13.149.254@KRBTEST.COM'} at 127.0.0.1:42388
I20260924 13:50:11.978632 28638 tablet_replica.cc:330] stopping tablet replica
I20260924 13:50:11.978754 28638 raft_consensus.cc:2229] T d5fb28bca78f4380bb28fa33322a35de P 666006d5e083435b9d59291a298b266c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 13:50:11.978824 28638 raft_consensus.cc:2258] T d5fb28bca78f4380bb28fa33322a35de P 666006d5e083435b9d59291a298b266c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 13:50:11.979128 28638 ts_tablet_manager.cc:1905] T d5fb28bca78f4380bb28fa33322a35de P 666006d5e083435b9d59291a298b266c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 13:50:11.981055 28638 ts_tablet_manager.cc:1918] T d5fb28bca78f4380bb28fa33322a35de P 666006d5e083435b9d59291a298b266c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 13:50:11.981122 28638 log.cc:1196] T d5fb28bca78f4380bb28fa33322a35de P 666006d5e083435b9d59291a298b266c: Deleting WAL directory at /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790257629041645-13911-0/minicluster-data/ts-1/wal/wals/d5fb28bca78f4380bb28fa33322a35de
I20260924 13:50:11.981317 28638 ts_tablet_manager.cc:1939] T d5fb28bca78f4380bb28fa33322a35de P 666006d5e083435b9d59291a298b266c: Deleting consensus metadata
I20260924 13:50:11.981814 28286 catalog_manager.cc:4841] TS 666006d5e083435b9d59291a298b266c (127.13.149.194:45113): tablet d5fb28bca78f4380bb28fa33322a35de (table test-table [id=19e3c3ba2a604baaac2c9e758cb207e9]) successfully deleted
W20260924 13:50:12.048084 28636 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 13:50:12.048300 28636 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 13:50:12.048322 28636 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 13:50:12.049456 28636 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 13:50:12.049495 28636 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 13:50:12.049521 28636 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.149.195
I20260924 13:50:12.050957 28636 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.149.212:45289
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790257629041645-13911-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790257629041645-13911-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.195.keytab
--principal=kudu/127.13.149.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.149.195:45095
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790257629041645-13911-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.149.195
--webserver_port=41563
--tserver_master_addrs=127.13.149.254:36075
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28636
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.149.195
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790257629041645-13911-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 12:29:37 UTC on ccycloud.quasar-iqzbxn.root.comops.site
build id 4670106
I20260924 13:50:12.051152 28636 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 13:50:12.051323 28636 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 13:50:12.056053 28646 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 13:50:12.056113 28644 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 13:50:12.056125 28643 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 13:50:12.056219 28636 server_base.cc:1034] running on GCE node
I20260924 13:50:12.056447 28636 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 13:50:12.056649 28636 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 13:50:12.057771 28636 hybrid_clock.cc:648] HybridClock initialized: now 1790257812057754 us; error 32 us; skew 500 ppm
Sep 24 13:50:12 dist-test-slave-sjl2 krb5kdc[27721](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257812, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.149.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 13:50:12.059993 28636 init.cc:377] Logged in from keytab as kudu/127.13.149.195@KRBTEST.COM (short username kudu)
I20260924 13:50:12.060415 28636 webserver.cc:460] Webserver started at http://127.13.149.195:41563/ using document root <none> and password file <none>
I20260924 13:50:12.060613 28636 fs_manager.cc:359] Metadata directory not provided
I20260924 13:50:12.060675 28636 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 13:50:12.061971 28636 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260924 13:50:12.062561 28653 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 13:50:12.062726 28636 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 13:50:12.062808 28636 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790257629041645-13911-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790257629041645-13911-0/minicluster-data/ts-2/wal
uuid: "8423e2366dc04ab8b01dc9d932e8ee75"
format_stamp: "Formatted at 2026-09-24 13:50:11 on dist-test-slave-sjl2"
server_key: "83f8cad45ec6d96e29788656175a6647"
server_key_iv: "6b65e4d52a42cc1cdac48a416fe21a38"
server_key_version: "encryptionkey@0"
I20260924 13:50:12.063169 28636 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790257629041645-13911-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790257629041645-13911-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790257629041645-13911-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 13:50:12.078487 28636 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 13:50:12.080878 28636 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 13:50:12.081048 28636 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 13:50:12.081265 28636 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 13:50:12.081569 28636 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 13:50:12.081606 28636 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 13:50:12.081635 28636 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 13:50:12.081660 28636 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 13:50:12.087133 28636 rpc_server.cc:307] RPC server started. Bound to: 127.13.149.195:45095
I20260924 13:50:12.087189 28766 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.149.195:45095 every 8 connection(s)
I20260924 13:50:12.087515 28636 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790257629041645-13911-0/minicluster-data/ts-2/data/info.pb
I20260924 13:50:12.090880 13911 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJN9WRF/build/release/bin/kudu as pid 28636
Sep 24 13:50:12 dist-test-slave-sjl2 krb5kdc[27721](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257812, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.149.195@KRBTEST.COM for kudu/127.13.149.254@KRBTEST.COM
I20260924 13:50:12.097177 28495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:50:12.088246 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.149.195:45361 (local address 127.13.149.254:36075)
0924 13:50:12.088306 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0924 13:50:12.088308 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:50:12.088510 (+ 202us) server_negotiation.cc:407] Connection header received
0924 13:50:12.089041 (+ 531us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:50:12.089042 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:50:12.089054 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:50:12.089124 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:50:12.090122 (+ 998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:50:12.090627 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:50:12.091901 (+ 1274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:50:12.092098 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:50:12.092153 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:50:12.094543 (+ 2390us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:50:12.094544 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:50:12.094544 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 13:50:12.094551 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:50:12.095838 (+ 1287us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:50:12.096262 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:50:12.096263 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:50:12.096263 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:50:12.096314 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:50:12.096551 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:50:12.096551 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:50:12.096551 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:50:12.096643 (+ 92us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:50:12.096700 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0924 13:50:12.097024 (+ 324us) server_negotiation.cc:299] Negotiation successful
0924 13:50:12.097106 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260924 13:50:12.097358 28769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:50:12.088231 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:36075 (local address 127.13.149.195:45361)
0924 13:50:12.088391 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0924 13:50:12.088426 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0924 13:50:12.088937 (+ 511us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:50:12.089220 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:50:12.089223 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:50:12.089839 (+ 616us) client_negotiation.cc:189] Negotiated authn=SASL
0924 13:50:12.090038 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:50:12.090044 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:50:12.090743 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:50:12.090744 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:50:12.091833 (+ 1089us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:50:12.091836 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:50:12.092164 (+ 328us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:50:12.092165 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:50:12.092287 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:50:12.092816 (+ 529us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 13:50:12.092826 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0924 13:50:12.094460 (+ 1634us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 13:50:12.095909 (+ 1449us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 13:50:12.095917 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 13:50:12.095919 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 13:50:12.096180 (+ 261us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 13:50:12.096364 (+ 184us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 13:50:12.096365 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 13:50:12.096365 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 13:50:12.096492 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 13:50:12.096704 (+ 212us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 13:50:12.096710 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 13:50:12.096868 (+ 158us) client_negotiation.cc:769] Sending connection context
0924 13:50:12.096986 (+ 118us) client_negotiation.cc:240] Negotiation successful
0924 13:50:12.097131 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":79,"threads_started":1}
I20260924 13:50:12.097856 28771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:50:12.091337 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48718 (local address 127.13.149.254:36075)
0924 13:50:12.091445 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0924 13:50:12.091447 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:50:12.091460 (+ 13us) server_negotiation.cc:407] Connection header received
0924 13:50:12.091525 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:50:12.091525 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:50:12.091537 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:50:12.091608 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:50:12.092105 (+ 497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:50:12.092643 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:50:12.093719 (+ 1076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:50:12.093909 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:50:12.093960 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:50:12.095247 (+ 1287us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:50:12.095249 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:50:12.095249 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 13:50:12.095257 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:50:12.096758 (+ 1501us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:50:12.097099 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:50:12.097100 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:50:12.097100 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:50:12.097157 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:50:12.097362 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:50:12.097363 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:50:12.097363 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:50:12.097485 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:50:12.097651 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0924 13:50:12.097691 (+ 40us) server_negotiation.cc:299] Negotiation successful
0924 13:50:12.097771 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86,"thread_start_us":56,"threads_started":1}
I20260924 13:50:12.097844 28767 heartbeater.cc:347] Connected to a master server at 127.13.149.254:36075
I20260924 13:50:12.097939 28767 heartbeater.cc:464] Registering TS with master...
I20260924 13:50:12.098119 28767 heartbeater.cc:511] Master 127.13.149.254:36075 requested a full tablet report, sending...
I20260924 13:50:12.098470 28300 ts_manager.cc:194] Registered new tserver with Master: 8423e2366dc04ab8b01dc9d932e8ee75 (127.13.149.195:45095)
I20260924 13:50:12.098834 28300 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.149.195@KRBTEST.COM'} at 127.13.149.195:45361
I20260924 13:50:12.109624 13911 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260924 13:50:12.112586 28771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:50:12.109976 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48732 (local address 127.13.149.254:36075)
0924 13:50:12.110063 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0924 13:50:12.110065 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:50:12.110121 (+ 56us) server_negotiation.cc:407] Connection header received
0924 13:50:12.110211 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:50:12.110212 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:50:12.110267 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:50:12.110337 (+ 70us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 13:50:12.110454 (+ 117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:50:12.110953 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:50:12.111723 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:50:12.111932 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:50:12.111991 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:50:12.112189 (+ 198us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 13:50:12.112313 (+ 124us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 13:50:12.112404 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0924 13:50:12.112487 (+ 83us) server_negotiation.cc:299] Negotiation successful
0924 13:50:12.112515 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260924 13:50:12.112999 28300 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:48732:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 13:50:12.114562 28300 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 13:50:12.116571 28424 tablet_service.cc:1467] Processing CreateTablet for tablet 86dac4433c7d404aabd4a777b6c89ccb (DEFAULT_TABLE table=test-table [id=84b4b09137444bd9a68c7c1644aa4414]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 13:50:12.116571 28565 tablet_service.cc:1467] Processing CreateTablet for tablet 86dac4433c7d404aabd4a777b6c89ccb (DEFAULT_TABLE table=test-table [id=84b4b09137444bd9a68c7c1644aa4414]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 13:50:12.116787 28565 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 86dac4433c7d404aabd4a777b6c89ccb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 13:50:12.116787 28424 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 86dac4433c7d404aabd4a777b6c89ccb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 13:50:12.118348 28381 tablet_bootstrap.cc:492] T 86dac4433c7d404aabd4a777b6c89ccb P 46b4fd024b654a5497f77167a74589cf: Bootstrap starting.
I20260924 13:50:12.118573 28523 tablet_bootstrap.cc:492] T 86dac4433c7d404aabd4a777b6c89ccb P 666006d5e083435b9d59291a298b266c: Bootstrap starting.
I20260924 13:50:12.118672 28381 tablet_bootstrap.cc:654] T 86dac4433c7d404aabd4a777b6c89ccb P 46b4fd024b654a5497f77167a74589cf: Neither blocks nor log segments found. Creating new log.
I20260924 13:50:12.118862 28523 tablet_bootstrap.cc:654] T 86dac4433c7d404aabd4a777b6c89ccb P 666006d5e083435b9d59291a298b266c: Neither blocks nor log segments found. Creating new log.
I20260924 13:50:12.119299 28381 tablet_bootstrap.cc:492] T 86dac4433c7d404aabd4a777b6c89ccb P 46b4fd024b654a5497f77167a74589cf: No bootstrap required, opened a new log
I20260924 13:50:12.119349 28381 ts_tablet_manager.cc:1397] T 86dac4433c7d404aabd4a777b6c89ccb P 46b4fd024b654a5497f77167a74589cf: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260924 13:50:12.119613 28523 tablet_bootstrap.cc:492] T 86dac4433c7d404aabd4a777b6c89ccb P 666006d5e083435b9d59291a298b266c: No bootstrap required, opened a new log
I20260924 13:50:12.119668 28523 ts_tablet_manager.cc:1397] T 86dac4433c7d404aabd4a777b6c89ccb P 666006d5e083435b9d59291a298b266c: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260924 13:50:12.119690 28381 raft_consensus.cc:348] T 86dac4433c7d404aabd4a777b6c89ccb P 46b4fd024b654a5497f77167a74589cf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8423e2366dc04ab8b01dc9d932e8ee75" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 45095 } } peers { permanent_uuid: "46b4fd024b654a5497f77167a74589cf" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 43429 } } peers { permanent_uuid: "666006d5e083435b9d59291a298b266c" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 45113 } }
I20260924 13:50:12.119781 28381 raft_consensus.cc:374] T 86dac4433c7d404aabd4a777b6c89ccb P 46b4fd024b654a5497f77167a74589cf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 13:50:12.119827 28381 raft_consensus.cc:729] T 86dac4433c7d404aabd4a777b6c89ccb P 46b4fd024b654a5497f77167a74589cf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 46b4fd024b654a5497f77167a74589cf, State: Initialized, Role: FOLLOWER
I20260924 13:50:12.119813 28523 raft_consensus.cc:348] T 86dac4433c7d404aabd4a777b6c89ccb P 666006d5e083435b9d59291a298b266c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8423e2366dc04ab8b01dc9d932e8ee75" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 45095 } } peers { permanent_uuid: "46b4fd024b654a5497f77167a74589cf" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 43429 } } peers { permanent_uuid: "666006d5e083435b9d59291a298b266c" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 45113 } }
I20260924 13:50:12.119868 28523 raft_consensus.cc:374] T 86dac4433c7d404aabd4a777b6c89ccb P 666006d5e083435b9d59291a298b266c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 13:50:12.119896 28523 raft_consensus.cc:729] T 86dac4433c7d404aabd4a777b6c89ccb P 666006d5e083435b9d59291a298b266c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 666006d5e083435b9d59291a298b266c, State: Initialized, Role: FOLLOWER
I20260924 13:50:12.119876 28381 consensus_queue.cc:260] T 86dac4433c7d404aabd4a777b6c89ccb P 46b4fd024b654a5497f77167a74589cf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8423e2366dc04ab8b01dc9d932e8ee75" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 45095 } } peers { permanent_uuid: "46b4fd024b654a5497f77167a74589cf" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 43429 } } peers { permanent_uuid: "666006d5e083435b9d59291a298b266c" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 45113 } }
I20260924 13:50:12.119933 28523 consensus_queue.cc:260] T 86dac4433c7d404aabd4a777b6c89ccb P 666006d5e083435b9d59291a298b266c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8423e2366dc04ab8b01dc9d932e8ee75" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 45095 } } peers { permanent_uuid: "46b4fd024b654a5497f77167a74589cf" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 43429 } } peers { permanent_uuid: "666006d5e083435b9d59291a298b266c" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 45113 } }
I20260924 13:50:12.119962 28381 ts_tablet_manager.cc:1428] T 86dac4433c7d404aabd4a777b6c89ccb P 46b4fd024b654a5497f77167a74589cf: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260924 13:50:12.119962 28499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:50:12.116305 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.195:45095 (local address 127.0.0.1:54620)
0924 13:50:12.116342 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0924 13:50:12.116353 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0924 13:50:12.116391 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:50:12.116768 (+ 377us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:50:12.116769 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:50:12.116771 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 13:50:12.116822 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:50:12.116823 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:50:12.117909 (+ 1086us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:50:12.117910 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:50:12.118851 (+ 941us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:50:12.118854 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:50:12.119713 (+ 859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:50:12.119714 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:50:12.119783 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:50:12.119812 (+ 29us) client_negotiation.cc:769] Sending connection context
0924 13:50:12.119847 (+ 35us) client_negotiation.cc:240] Negotiation successful
0924 13:50:12.119895 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":19}
I20260924 13:50:12.120018 28523 ts_tablet_manager.cc:1428] T 86dac4433c7d404aabd4a777b6c89ccb P 666006d5e083435b9d59291a298b266c: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260924 13:50:12.120347 28773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:50:12.116417 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54620 (local address 127.13.149.195:45095)
0924 13:50:12.116596 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0924 13:50:12.116601 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 13:50:12.116610 (+ 9us) server_negotiation.cc:407] Connection header received
0924 13:50:12.116645 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:50:12.116647 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:50:12.116661 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:50:12.116892 (+ 231us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 13:50:12.116977 (+ 85us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:50:12.117820 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:50:12.118924 (+ 1104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:50:12.119616 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:50:12.120113 (+ 497us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:50:12.120183 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0924 13:50:12.120210 (+ 27us) server_negotiation.cc:299] Negotiation successful
0924 13:50:12.120264 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":146,"thread_start_us":99,"threads_started":1}
I20260924 13:50:12.121088 28701 tablet_service.cc:1467] Processing CreateTablet for tablet 86dac4433c7d404aabd4a777b6c89ccb (DEFAULT_TABLE table=test-table [id=84b4b09137444bd9a68c7c1644aa4414]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 13:50:12.121372 28701 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 86dac4433c7d404aabd4a777b6c89ccb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 13:50:12.124078 28776 tablet_bootstrap.cc:492] T 86dac4433c7d404aabd4a777b6c89ccb P 8423e2366dc04ab8b01dc9d932e8ee75: Bootstrap starting.
I20260924 13:50:12.124704 28776 tablet_bootstrap.cc:654] T 86dac4433c7d404aabd4a777b6c89ccb P 8423e2366dc04ab8b01dc9d932e8ee75: Neither blocks nor log segments found. Creating new log.
I20260924 13:50:12.125237 28776 log.cc:824] T 86dac4433c7d404aabd4a777b6c89ccb P 8423e2366dc04ab8b01dc9d932e8ee75: Log is configured to *not* fsync() on all Append() calls
I20260924 13:50:12.125921 28776 tablet_bootstrap.cc:492] T 86dac4433c7d404aabd4a777b6c89ccb P 8423e2366dc04ab8b01dc9d932e8ee75: No bootstrap required, opened a new log
I20260924 13:50:12.125982 28776 ts_tablet_manager.cc:1397] T 86dac4433c7d404aabd4a777b6c89ccb P 8423e2366dc04ab8b01dc9d932e8ee75: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260924 13:50:12.127445 28776 raft_consensus.cc:348] T 86dac4433c7d404aabd4a777b6c89ccb P 8423e2366dc04ab8b01dc9d932e8ee75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8423e2366dc04ab8b01dc9d932e8ee75" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 45095 } } peers { permanent_uuid: "46b4fd024b654a5497f77167a74589cf" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 43429 } } peers { permanent_uuid: "666006d5e083435b9d59291a298b266c" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 45113 } }
I20260924 13:50:12.127583 28776 raft_consensus.cc:374] T 86dac4433c7d404aabd4a777b6c89ccb P 8423e2366dc04ab8b01dc9d932e8ee75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 13:50:12.127604 28776 raft_consensus.cc:729] T 86dac4433c7d404aabd4a777b6c89ccb P 8423e2366dc04ab8b01dc9d932e8ee75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8423e2366dc04ab8b01dc9d932e8ee75, State: Initialized, Role: FOLLOWER
I20260924 13:50:12.127696 28776 consensus_queue.cc:260] T 86dac4433c7d404aabd4a777b6c89ccb P 8423e2366dc04ab8b01dc9d932e8ee75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8423e2366dc04ab8b01dc9d932e8ee75" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 45095 } } peers { permanent_uuid: "46b4fd024b654a5497f77167a74589cf" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 43429 } } peers { permanent_uuid: "666006d5e083435b9d59291a298b266c" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 45113 } }
I20260924 13:50:12.127926 28776 ts_tablet_manager.cc:1428] T 86dac4433c7d404aabd4a777b6c89ccb P 8423e2366dc04ab8b01dc9d932e8ee75: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260924 13:50:12.127990 28767 heartbeater.cc:503] Master 127.13.149.254:36075 was elected leader, sending a full tablet report...
I20260924 13:50:12.169302 28778 raft_consensus.cc:482] T 86dac4433c7d404aabd4a777b6c89ccb P 8423e2366dc04ab8b01dc9d932e8ee75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 13:50:12.169420 28778 raft_consensus.cc:504] T 86dac4433c7d404aabd4a777b6c89ccb P 8423e2366dc04ab8b01dc9d932e8ee75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8423e2366dc04ab8b01dc9d932e8ee75" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 45095 } } peers { permanent_uuid: "46b4fd024b654a5497f77167a74589cf" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 43429 } } peers { permanent_uuid: "666006d5e083435b9d59291a298b266c" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 45113 } }
I20260924 13:50:12.169788 28778 leader_election.cc:288] T 86dac4433c7d404aabd4a777b6c89ccb P 8423e2366dc04ab8b01dc9d932e8ee75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 46b4fd024b654a5497f77167a74589cf (127.13.149.193:43429), 666006d5e083435b9d59291a298b266c (127.13.149.194:45113)
I20260924 13:50:12.172611 28637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:50:12.170063 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.149.195:47395 (local address 127.13.149.194:45113)
0924 13:50:12.170115 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0924 13:50:12.170118 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 13:50:12.170127 (+ 9us) server_negotiation.cc:407] Connection header received
0924 13:50:12.170162 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:50:12.170163 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:50:12.170192 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:50:12.170266 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 13:50:12.170421 (+ 155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:50:12.170929 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:50:12.171836 (+ 907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:50:12.172260 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:50:12.172333 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:50:12.172415 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0924 13:50:12.172490 (+ 75us) server_negotiation.cc:299] Negotiation successful
0924 13:50:12.172533 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260924 13:50:12.172633 28769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:50:12.169968 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.194:45113 (local address 127.13.149.195:47395)
0924 13:50:12.170020 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0924 13:50:12.170031 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0924 13:50:12.170071 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:50:12.170279 (+ 208us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:50:12.170281 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:50:12.170283 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 13:50:12.170353 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:50:12.170355 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:50:12.171009 (+ 654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:50:12.171010 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:50:12.171742 (+ 732us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:50:12.171744 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:50:12.172337 (+ 593us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:50:12.172338 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:50:12.172419 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:50:12.172423 (+ 4us) client_negotiation.cc:769] Sending connection context
0924 13:50:12.172480 (+ 57us) client_negotiation.cc:240] Negotiation successful
0924 13:50:12.172524 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34}
I20260924 13:50:12.172649 28500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:50:12.170035 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.149.195:51863 (local address 127.13.149.193:43429)
0924 13:50:12.170074 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0924 13:50:12.170076 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:50:12.170249 (+ 173us) server_negotiation.cc:407] Connection header received
0924 13:50:12.170302 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:50:12.170303 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:50:12.170328 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:50:12.170382 (+ 54us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 13:50:12.170521 (+ 139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:50:12.171014 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:50:12.171910 (+ 896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:50:12.172325 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:50:12.172389 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:50:12.172457 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0924 13:50:12.172539 (+ 82us) server_negotiation.cc:299] Negotiation successful
0924 13:50:12.172582 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260924 13:50:12.173005 28779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:50:12.169982 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.193:43429 (local address 127.13.149.195:51863)
0924 13:50:12.170186 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0924 13:50:12.170193 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0924 13:50:12.170248 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:50:12.170395 (+ 147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:50:12.170396 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:50:12.170397 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 13:50:12.170466 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:50:12.170468 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:50:12.171091 (+ 623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:50:12.171092 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:50:12.171829 (+ 737us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:50:12.171831 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:50:12.172405 (+ 574us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:50:12.172406 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:50:12.172474 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:50:12.172478 (+ 4us) client_negotiation.cc:769] Sending connection context
0924 13:50:12.172530 (+ 52us) client_negotiation.cc:240] Negotiation successful
0924 13:50:12.172567 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":160,"threads_started":1}
I20260924 13:50:12.173125 28585 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "86dac4433c7d404aabd4a777b6c89ccb" candidate_uuid: "8423e2366dc04ab8b01dc9d932e8ee75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "666006d5e083435b9d59291a298b266c" is_pre_election: true
I20260924 13:50:12.173290 28585 raft_consensus.cc:2454] T 86dac4433c7d404aabd4a777b6c89ccb P 666006d5e083435b9d59291a298b266c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8423e2366dc04ab8b01dc9d932e8ee75 in term 0.
I20260924 13:50:12.173349 28430 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "86dac4433c7d404aabd4a777b6c89ccb" candidate_uuid: "8423e2366dc04ab8b01dc9d932e8ee75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "46b4fd024b654a5497f77167a74589cf" is_pre_election: true
I20260924 13:50:12.173507 28430 raft_consensus.cc:2454] T 86dac4433c7d404aabd4a777b6c89ccb P 46b4fd024b654a5497f77167a74589cf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8423e2366dc04ab8b01dc9d932e8ee75 in term 0.
I20260924 13:50:12.173523 28655 leader_election.cc:302] T 86dac4433c7d404aabd4a777b6c89ccb P 8423e2366dc04ab8b01dc9d932e8ee75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 666006d5e083435b9d59291a298b266c, 8423e2366dc04ab8b01dc9d932e8ee75; no voters:
I20260924 13:50:12.173635 28778 raft_consensus.cc:2781] T 86dac4433c7d404aabd4a777b6c89ccb P 8423e2366dc04ab8b01dc9d932e8ee75 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 13:50:12.173709 28778 raft_consensus.cc:482] T 86dac4433c7d404aabd4a777b6c89ccb P 8423e2366dc04ab8b01dc9d932e8ee75 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 13:50:12.173736 28778 raft_consensus.cc:3037] T 86dac4433c7d404aabd4a777b6c89ccb P 8423e2366dc04ab8b01dc9d932e8ee75 [term 0 FOLLOWER]: Advancing to term 1
I20260924 13:50:12.174576 28778 raft_consensus.cc:504] T 86dac4433c7d404aabd4a777b6c89ccb P 8423e2366dc04ab8b01dc9d932e8ee75 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8423e2366dc04ab8b01dc9d932e8ee75" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 45095 } } peers { permanent_uuid: "46b4fd024b654a5497f77167a74589cf" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 43429 } } peers { permanent_uuid: "666006d5e083435b9d59291a298b266c" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 45113 } }
I20260924 13:50:12.174736 28778 leader_election.cc:288] T 86dac4433c7d404aabd4a777b6c89ccb P 8423e2366dc04ab8b01dc9d932e8ee75 [CANDIDATE]: Term 1 election: Requested vote from peers 46b4fd024b654a5497f77167a74589cf (127.13.149.193:43429), 666006d5e083435b9d59291a298b266c (127.13.149.194:45113)
I20260924 13:50:12.174883 28430 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "86dac4433c7d404aabd4a777b6c89ccb" candidate_uuid: "8423e2366dc04ab8b01dc9d932e8ee75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "46b4fd024b654a5497f77167a74589cf"
I20260924 13:50:12.174964 28430 raft_consensus.cc:3037] T 86dac4433c7d404aabd4a777b6c89ccb P 46b4fd024b654a5497f77167a74589cf [term 0 FOLLOWER]: Advancing to term 1
I20260924 13:50:12.175084 28585 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "86dac4433c7d404aabd4a777b6c89ccb" candidate_uuid: "8423e2366dc04ab8b01dc9d932e8ee75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "666006d5e083435b9d59291a298b266c"
I20260924 13:50:12.175222 28585 raft_consensus.cc:3037] T 86dac4433c7d404aabd4a777b6c89ccb P 666006d5e083435b9d59291a298b266c [term 0 FOLLOWER]: Advancing to term 1
I20260924 13:50:12.175779 28430 raft_consensus.cc:2454] T 86dac4433c7d404aabd4a777b6c89ccb P 46b4fd024b654a5497f77167a74589cf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8423e2366dc04ab8b01dc9d932e8ee75 in term 1.
I20260924 13:50:12.175899 28585 raft_consensus.cc:2454] T 86dac4433c7d404aabd4a777b6c89ccb P 666006d5e083435b9d59291a298b266c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8423e2366dc04ab8b01dc9d932e8ee75 in term 1.
I20260924 13:50:12.175966 28657 leader_election.cc:302] T 86dac4433c7d404aabd4a777b6c89ccb P 8423e2366dc04ab8b01dc9d932e8ee75 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46b4fd024b654a5497f77167a74589cf, 8423e2366dc04ab8b01dc9d932e8ee75; no voters:
I20260924 13:50:12.176095 28778 raft_consensus.cc:2781] T 86dac4433c7d404aabd4a777b6c89ccb P 8423e2366dc04ab8b01dc9d932e8ee75 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 13:50:12.176278 28778 raft_consensus.cc:686] T 86dac4433c7d404aabd4a777b6c89ccb P 8423e2366dc04ab8b01dc9d932e8ee75 [term 1 LEADER]: Becoming Leader. State: Replica: 8423e2366dc04ab8b01dc9d932e8ee75, State: Running, Role: LEADER
I20260924 13:50:12.176384 28778 consensus_queue.cc:237] T 86dac4433c7d404aabd4a777b6c89ccb P 8423e2366dc04ab8b01dc9d932e8ee75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8423e2366dc04ab8b01dc9d932e8ee75" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 45095 } } peers { permanent_uuid: "46b4fd024b654a5497f77167a74589cf" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 43429 } } peers { permanent_uuid: "666006d5e083435b9d59291a298b266c" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 45113 } }
I20260924 13:50:12.177248 28301 catalog_manager.cc:5495] T 86dac4433c7d404aabd4a777b6c89ccb P 8423e2366dc04ab8b01dc9d932e8ee75 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8423e2366dc04ab8b01dc9d932e8ee75 (127.13.149.195). New cstate: current_term: 1 leader_uuid: "8423e2366dc04ab8b01dc9d932e8ee75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8423e2366dc04ab8b01dc9d932e8ee75" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 45095 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "46b4fd024b654a5497f77167a74589cf" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 43429 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "666006d5e083435b9d59291a298b266c" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 45113 } health_report { overall_health: UNKNOWN } } }
I20260924 13:50:12.194380 28773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:50:12.191969 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54632 (local address 127.13.149.195:45095)
0924 13:50:12.192018 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0924 13:50:12.192020 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:50:12.192030 (+ 10us) server_negotiation.cc:407] Connection header received
0924 13:50:12.192071 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:50:12.192073 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:50:12.192089 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:50:12.192145 (+ 56us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 13:50:12.192283 (+ 138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:50:12.192783 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:50:12.193547 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:50:12.193816 (+ 269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:50:12.193874 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:50:12.194040 (+ 166us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 13:50:12.194145 (+ 105us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 13:50:12.194217 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0924 13:50:12.194256 (+ 39us) server_negotiation.cc:299] Negotiation successful
0924 13:50:12.194279 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260924 13:50:12.196213 28430 raft_consensus.cc:1261] T 86dac4433c7d404aabd4a777b6c89ccb P 46b4fd024b654a5497f77167a74589cf [term 1 FOLLOWER]: Refusing update from remote peer 8423e2366dc04ab8b01dc9d932e8ee75: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 13:50:12.196205 28585 raft_consensus.cc:1261] T 86dac4433c7d404aabd4a777b6c89ccb P 666006d5e083435b9d59291a298b266c [term 1 FOLLOWER]: Refusing update from remote peer 8423e2366dc04ab8b01dc9d932e8ee75: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 13:50:12.196554 28778 consensus_queue.cc:1035] T 86dac4433c7d404aabd4a777b6c89ccb P 8423e2366dc04ab8b01dc9d932e8ee75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "666006d5e083435b9d59291a298b266c" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 45113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 13:50:12.196694 28780 consensus_queue.cc:1035] T 86dac4433c7d404aabd4a777b6c89ccb P 8423e2366dc04ab8b01dc9d932e8ee75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46b4fd024b654a5497f77167a74589cf" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 43429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 13:50:12.199481 28785 mvcc.cc:205] Tried to move back new op lower bound from 7332895998752985088 to 7332895998675243008. Current Snapshot: MvccSnapshot[applied={T|T < 7332895998752985088}]
I20260924 13:50:12.200804 28300 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:48732:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 13:50:12.200896 28300 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:48732:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 13:50:12.201618 28300 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ffaff00f2fee4d03bb1f71c504fabd7f: Sending DeleteTablet for 3 replicas of tablet 86dac4433c7d404aabd4a777b6c89ccb
I20260924 13:50:12.201970 28565 tablet_service.cc:1514] Processing DeleteTablet for tablet 86dac4433c7d404aabd4a777b6c89ccb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 13:50:12 UTC) from {username='kudu', principal='kudu/127.13.149.254@KRBTEST.COM'} at 127.0.0.1:42388
I20260924 13:50:12.201970 13911 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJN9WRF/build/release/bin/kudu with pid 28339
I20260924 13:50:12.201960 28701 tablet_service.cc:1514] Processing DeleteTablet for tablet 86dac4433c7d404aabd4a777b6c89ccb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 13:50:12 UTC) from {username='kudu', principal='kudu/127.13.149.254@KRBTEST.COM'} at 127.0.0.1:54620
I20260924 13:50:12.202059 28424 tablet_service.cc:1514] Processing DeleteTablet for tablet 86dac4433c7d404aabd4a777b6c89ccb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 13:50:12 UTC) from {username='kudu', principal='kudu/127.13.149.254@KRBTEST.COM'} at 127.0.0.1:46944
I20260924 13:50:12.202142 28638 tablet_replica.cc:330] T 86dac4433c7d404aabd4a777b6c89ccb P 666006d5e083435b9d59291a298b266c: stopping tablet replica
I20260924 13:50:12.202183 28638 raft_consensus.cc:2229] T 86dac4433c7d404aabd4a777b6c89ccb P 666006d5e083435b9d59291a298b266c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 13:50:12.202215 28638 raft_consensus.cc:2258] T 86dac4433c7d404aabd4a777b6c89ccb P 666006d5e083435b9d59291a298b266c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 13:50:12.203179 28638 ts_tablet_manager.cc:1905] T 86dac4433c7d404aabd4a777b6c89ccb P 666006d5e083435b9d59291a298b266c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 13:50:12.204952 28638 ts_tablet_manager.cc:1918] T 86dac4433c7d404aabd4a777b6c89ccb P 666006d5e083435b9d59291a298b266c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 13:50:12.205010 28638 log.cc:1196] T 86dac4433c7d404aabd4a777b6c89ccb P 666006d5e083435b9d59291a298b266c: Deleting WAL directory at /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790257629041645-13911-0/minicluster-data/ts-1/wal/wals/86dac4433c7d404aabd4a777b6c89ccb
I20260924 13:50:12.205196 28638 ts_tablet_manager.cc:1939] T 86dac4433c7d404aabd4a777b6c89ccb P 666006d5e083435b9d59291a298b266c: Deleting consensus metadata
I20260924 13:50:12.205549 28286 catalog_manager.cc:4841] TS 666006d5e083435b9d59291a298b266c (127.13.149.194:45113): tablet 86dac4433c7d404aabd4a777b6c89ccb (table test-table [id=84b4b09137444bd9a68c7c1644aa4414]) successfully deleted
W20260924 13:50:12.207989 28288 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.13.149.193:43429 (error 108)
I20260924 13:50:12.208348 13911 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJN9WRF/build/release/bin/kudu with pid 28497
I20260924 13:50:12.208503 28499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:50:12.208371 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.193:43429 (local address 127.0.0.1:46950)
0924 13:50:12.208425 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0924 13:50:12.208470 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.149.193:43429: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260924 13:50:12.208508 28792 tablet_replica.cc:330] T 86dac4433c7d404aabd4a777b6c89ccb P 8423e2366dc04ab8b01dc9d932e8ee75: stopping tablet replica
I20260924 13:50:12.208626 28792 raft_consensus.cc:2229] T 86dac4433c7d404aabd4a777b6c89ccb P 8423e2366dc04ab8b01dc9d932e8ee75 [term 1 LEADER]: Raft consensus shutting down.
I20260924 13:50:12.208757 28792 raft_consensus.cc:2258] T 86dac4433c7d404aabd4a777b6c89ccb P 8423e2366dc04ab8b01dc9d932e8ee75 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260924 13:50:12.209429 28288 catalog_manager.cc:4584] TS 46b4fd024b654a5497f77167a74589cf (127.13.149.193:43429): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 86dac4433c7d404aabd4a777b6c89ccb: Network error: Client connection negotiation failed: client connection to 127.13.149.193:43429: connect: Connection refused (error 111)
I20260924 13:50:12.210188 28792 ts_tablet_manager.cc:1905] T 86dac4433c7d404aabd4a777b6c89ccb P 8423e2366dc04ab8b01dc9d932e8ee75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 13:50:12.211875 28792 ts_tablet_manager.cc:1918] T 86dac4433c7d404aabd4a777b6c89ccb P 8423e2366dc04ab8b01dc9d932e8ee75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 13:50:12.211946 28792 log.cc:1196] T 86dac4433c7d404aabd4a777b6c89ccb P 8423e2366dc04ab8b01dc9d932e8ee75: Deleting WAL directory at /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790257629041645-13911-0/minicluster-data/ts-2/wal/wals/86dac4433c7d404aabd4a777b6c89ccb
I20260924 13:50:12.212179 28792 ts_tablet_manager.cc:1939] T 86dac4433c7d404aabd4a777b6c89ccb P 8423e2366dc04ab8b01dc9d932e8ee75: Deleting consensus metadata
I20260924 13:50:12.212613 28288 catalog_manager.cc:4841] TS 8423e2366dc04ab8b01dc9d932e8ee75 (127.13.149.195:45095): tablet 86dac4433c7d404aabd4a777b6c89ccb (table test-table [id=84b4b09137444bd9a68c7c1644aa4414]) successfully deleted
I20260924 13:50:12.214442 13911 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJN9WRF/build/release/bin/kudu with pid 28636
I20260924 13:50:12.219532 13911 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJN9WRF/build/release/bin/kudu with pid 28268
I20260924 13:50:12.223971 13911 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJN9WRF/build/release/bin/kudu
/tmp/dist-test-taskJN9WRF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790257629041645-13911-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790257629041645-13911-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790257629041645-13911-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790257629041645-13911-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.149.254:36075
--webserver_interface=127.13.149.254
--webserver_port=44463
--keytab_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.254.keytab
--principal=kudu/127.13.149.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.149.212:45289
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.149.254:36075
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790257629041645-13911-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790257629041645-13911-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790257629041645-13911-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 13:50:12.304106 28794 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 13:50:12.304306 28794 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 13:50:12.304328 28794 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 13:50:12.305481 28794 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 13:50:12.305521 28794 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 13:50:12.305532 28794 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 13:50:12.305543 28794 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 13:50:12.305554 28794 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 13:50:12.307003 28794 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.149.212:45289
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790257629041645-13911-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790257629041645-13911-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.149.254:36075
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.254.keytab
--principal=kudu/127.13.149.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.149.254:36075
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790257629041645-13911-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.149.254
--webserver_port=44463
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28794
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790257629041645-13911-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 12:29:37 UTC on ccycloud.quasar-iqzbxn.root.comops.site
build id 4670106
I20260924 13:50:12.307199 28794 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 13:50:12.307382 28794 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 13:50:12.312162 28802 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 13:50:12.312153 28799 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 13:50:12.312249 28794 server_base.cc:1034] running on GCE node
W20260924 13:50:12.312139 28800 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 13:50:12.312642 28794 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 13:50:12.312955 28794 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 13:50:12.314126 28794 hybrid_clock.cc:648] HybridClock initialized: now 1790257812314111 us; error 28 us; skew 500 ppm
Sep 24 13:50:12 dist-test-slave-sjl2 krb5kdc[27721](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257812, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.149.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 13:50:12.316641 28794 init.cc:377] Logged in from keytab as kudu/127.13.149.254@KRBTEST.COM (short username kudu)
I20260924 13:50:12.317157 28794 webserver.cc:460] Webserver started at http://127.13.149.254:44463/ using document root <none> and password file <none>
I20260924 13:50:12.317379 28794 fs_manager.cc:359] Metadata directory not provided
I20260924 13:50:12.317428 28794 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 13:50:12.318708 28794 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260924 13:50:12.319412 28809 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 13:50:12.319579 28794 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260924 13:50:12.319661 28794 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790257629041645-13911-0/minicluster-data/master-0/data,/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790257629041645-13911-0/minicluster-data/master-0/wal
uuid: "ffaff00f2fee4d03bb1f71c504fabd7f"
format_stamp: "Formatted at 2026-09-24 13:50:10 on dist-test-slave-sjl2"
server_key: "b5ae1bf96c07654927cebe93e7a64d6e"
server_key_iv: "0efeda2723646de6a9a598a3753139cd"
server_key_version: "encryptionkey@0"
I20260924 13:50:12.320015 28794 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790257629041645-13911-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790257629041645-13911-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790257629041645-13911-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 13:50:12.344404 28794 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 13:50:12.346362 28794 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 13:50:12.346504 28794 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 13:50:12.350229 28794 rpc_server.cc:307] RPC server started. Bound to: 127.13.149.254:36075
I20260924 13:50:12.350265 28861 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.149.254:36075 every 8 connection(s)
I20260924 13:50:12.350610 28794 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790257629041645-13911-0/minicluster-data/master-0/data/info.pb
I20260924 13:50:12.352190 28862 sys_catalog.cc:263] Verifying existing consensus state
I20260924 13:50:12.352896 28862 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ffaff00f2fee4d03bb1f71c504fabd7f: Bootstrap starting.
I20260924 13:50:12.358719 13911 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJN9WRF/build/release/bin/kudu as pid 28794
I20260924 13:50:12.359009 13911 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJN9WRF/build/release/bin/kudu
/tmp/dist-test-taskJN9WRF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790257629041645-13911-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790257629041645-13911-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790257629041645-13911-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790257629041645-13911-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.149.193:43429
--local_ip_for_outbound_sockets=127.13.149.193
--tserver_master_addrs=127.13.149.254:36075
--webserver_port=43713
--webserver_interface=127.13.149.193
--keytab_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.193.keytab
--principal=kudu/127.13.149.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.149.212:45289
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790257629041645-13911-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790257629041645-13911-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790257629041645-13911-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260924 13:50:12.361449 28862 log.cc:824] T 00000000000000000000000000000000 P ffaff00f2fee4d03bb1f71c504fabd7f: Log is configured to *not* fsync() on all Append() calls
I20260924 13:50:12.363641 28862 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ffaff00f2fee4d03bb1f71c504fabd7f: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260924 13:50:12.363929 28862 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ffaff00f2fee4d03bb1f71c504fabd7f: Bootstrap complete.
I20260924 13:50:12.366081 28862 raft_consensus.cc:348] T 00000000000000000000000000000000 P ffaff00f2fee4d03bb1f71c504fabd7f [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ffaff00f2fee4d03bb1f71c504fabd7f" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 36075 } }
I20260924 13:50:12.366336 28862 raft_consensus.cc:729] T 00000000000000000000000000000000 P ffaff00f2fee4d03bb1f71c504fabd7f [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: ffaff00f2fee4d03bb1f71c504fabd7f, State: Initialized, Role: FOLLOWER
I20260924 13:50:12.366497 28862 consensus_queue.cc:260] T 00000000000000000000000000000000 P ffaff00f2fee4d03bb1f71c504fabd7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ffaff00f2fee4d03bb1f71c504fabd7f" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 36075 } }
I20260924 13:50:12.366575 28862 raft_consensus.cc:388] T 00000000000000000000000000000000 P ffaff00f2fee4d03bb1f71c504fabd7f [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 13:50:12.366623 28862 raft_consensus.cc:482] T 00000000000000000000000000000000 P ffaff00f2fee4d03bb1f71c504fabd7f [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 13:50:12.366670 28862 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ffaff00f2fee4d03bb1f71c504fabd7f [term 2 FOLLOWER]: Advancing to term 3
I20260924 13:50:12.367654 28862 raft_consensus.cc:504] T 00000000000000000000000000000000 P ffaff00f2fee4d03bb1f71c504fabd7f [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ffaff00f2fee4d03bb1f71c504fabd7f" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 36075 } }
I20260924 13:50:12.367749 28862 leader_election.cc:302] T 00000000000000000000000000000000 P ffaff00f2fee4d03bb1f71c504fabd7f [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ffaff00f2fee4d03bb1f71c504fabd7f; no voters:
I20260924 13:50:12.367929 28862 leader_election.cc:288] T 00000000000000000000000000000000 P ffaff00f2fee4d03bb1f71c504fabd7f [CANDIDATE]: Term 3 election: Requested vote from peers
I20260924 13:50:12.367998 28866 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ffaff00f2fee4d03bb1f71c504fabd7f [term 3 FOLLOWER]: Leader election won for term 3
I20260924 13:50:12.368151 28866 raft_consensus.cc:686] T 00000000000000000000000000000000 P ffaff00f2fee4d03bb1f71c504fabd7f [term 3 LEADER]: Becoming Leader. State: Replica: ffaff00f2fee4d03bb1f71c504fabd7f, State: Running, Role: LEADER
I20260924 13:50:12.368192 28862 sys_catalog.cc:564] T 00000000000000000000000000000000 P ffaff00f2fee4d03bb1f71c504fabd7f [sys.catalog]: configured and running, proceeding with master startup.
I20260924 13:50:12.368256 28866 consensus_queue.cc:237] T 00000000000000000000000000000000 P ffaff00f2fee4d03bb1f71c504fabd7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ffaff00f2fee4d03bb1f71c504fabd7f" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 36075 } }
I20260924 13:50:12.368553 28866 sys_catalog.cc:455] T 00000000000000000000000000000000 P ffaff00f2fee4d03bb1f71c504fabd7f [sys.catalog]: SysCatalogTable state changed. Reason: New leader ffaff00f2fee4d03bb1f71c504fabd7f. Latest consensus state: current_term: 3 leader_uuid: "ffaff00f2fee4d03bb1f71c504fabd7f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ffaff00f2fee4d03bb1f71c504fabd7f" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 36075 } } }
I20260924 13:50:12.368618 28866 sys_catalog.cc:458] T 00000000000000000000000000000000 P ffaff00f2fee4d03bb1f71c504fabd7f [sys.catalog]: This master's current role is: LEADER
I20260924 13:50:12.368736 28867 sys_catalog.cc:455] T 00000000000000000000000000000000 P ffaff00f2fee4d03bb1f71c504fabd7f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "ffaff00f2fee4d03bb1f71c504fabd7f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ffaff00f2fee4d03bb1f71c504fabd7f" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 36075 } } }
I20260924 13:50:12.368818 28867 sys_catalog.cc:458] T 00000000000000000000000000000000 P ffaff00f2fee4d03bb1f71c504fabd7f [sys.catalog]: This master's current role is: LEADER
I20260924 13:50:12.368808 28872 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 13:50:12.369635 28872 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 13:50:12.369748 28872 catalog_manager.cc:1259] Loaded cluster ID: da0001853c6b4ac9b2a8cdec412b8ea4
I20260924 13:50:12.370015 28872 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260924 13:50:12.370287 28872 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7fde024549ad google::LogMessage::Fail() at ??:0
@ 0x7fde0245788a google::LogMessage::SendToLog() at ??:0
@ 0x7fde024544de google::LogMessage::Flush() at ??:0
@ 0x7fde02455a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7fde0a3750f1 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7fde0a37e8b9 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7fde043220d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7fde04317954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7fde0517b6db start_thread at ??:0
@ 0x7fde00f5261f clone at ??:0
W20260924 13:50:12.450950 28864 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 13:50:12.451180 28864 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 13:50:12.451269 28864 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 13:50:12.452555 28864 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 13:50:12.452600 28864 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 13:50:12.452627 28864 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.149.193
I20260924 13:50:12.454222 28864 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.149.212:45289
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790257629041645-13911-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790257629041645-13911-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.193.keytab
--principal=kudu/127.13.149.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.149.193:43429
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790257629041645-13911-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.149.193
--webserver_port=43713
--tserver_master_addrs=127.13.149.254:36075
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28864
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.149.193
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790257629041645-13911-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 12:29:37 UTC on ccycloud.quasar-iqzbxn.root.comops.site
build id 4670106
I20260924 13:50:12.454459 28864 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 13:50:12.454681 28864 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 13:50:12.459582 28898 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 13:50:12.459606 28899 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 13:50:12.459877 28864 server_base.cc:1034] running on GCE node
W20260924 13:50:12.459619 28901 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 13:50:12.460156 28864 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 13:50:12.460453 28864 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 13:50:12.461632 28864 hybrid_clock.cc:648] HybridClock initialized: now 1790257812461595 us; error 49 us; skew 500 ppm
Sep 24 13:50:12 dist-test-slave-sjl2 krb5kdc[27721](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257812, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.149.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 13:50:12.464109 28864 init.cc:377] Logged in from keytab as kudu/127.13.149.193@KRBTEST.COM (short username kudu)
I20260924 13:50:12.464582 28864 webserver.cc:460] Webserver started at http://127.13.149.193:43713/ using document root <none> and password file <none>
I20260924 13:50:12.464810 28864 fs_manager.cc:359] Metadata directory not provided
I20260924 13:50:12.464854 28864 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 13:50:12.466176 28864 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260924 13:50:12.466832 28908 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 13:50:12.467007 28864 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 13:50:12.467132 28864 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790257629041645-13911-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790257629041645-13911-0/minicluster-data/ts-0/wal
uuid: "46b4fd024b654a5497f77167a74589cf"
format_stamp: "Formatted at 2026-09-24 13:50:11 on dist-test-slave-sjl2"
server_key: "421ffd5752f236310d95d01d0d561f72"
server_key_iv: "3279b4f8ab63fc4df9ce5e8ad6474835"
server_key_version: "encryptionkey@0"
I20260924 13:50:12.467578 28864 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790257629041645-13911-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790257629041645-13911-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790257629041645-13911-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 13:50:12.484222 28864 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 13:50:12.486799 28864 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 13:50:12.486954 28864 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 13:50:12.487216 28864 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 13:50:12.487753 28915 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260924 13:50:12.489332 28864 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260924 13:50:12.489394 28864 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260924 13:50:12.489429 28864 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260924 13:50:12.490299 28864 ts_tablet_manager.cc:610] Registered 1 tablets
I20260924 13:50:12.490345 28864 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260924 13:50:12.490449 28915 tablet_bootstrap.cc:492] T 86dac4433c7d404aabd4a777b6c89ccb P 46b4fd024b654a5497f77167a74589cf: Bootstrap starting.
I20260924 13:50:12.497560 28864 rpc_server.cc:307] RPC server started. Bound to: 127.13.149.193:43429
I20260924 13:50:12.497807 29022 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.149.193:43429 every 8 connection(s)
I20260924 13:50:12.498037 28864 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790257629041645-13911-0/minicluster-data/ts-0/data/info.pb
I20260924 13:50:12.501137 28915 log.cc:824] T 86dac4433c7d404aabd4a777b6c89ccb P 46b4fd024b654a5497f77167a74589cf: Log is configured to *not* fsync() on all Append() calls
I20260924 13:50:12.503228 28915 tablet_bootstrap.cc:492] T 86dac4433c7d404aabd4a777b6c89ccb P 46b4fd024b654a5497f77167a74589cf: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260924 13:50:12.503491 28915 tablet_bootstrap.cc:492] T 86dac4433c7d404aabd4a777b6c89ccb P 46b4fd024b654a5497f77167a74589cf: Bootstrap complete.
I20260924 13:50:12.504168 13911 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJN9WRF/build/release/bin/kudu as pid 28864
I20260924 13:50:12.504262 28915 ts_tablet_manager.cc:1397] T 86dac4433c7d404aabd4a777b6c89ccb P 46b4fd024b654a5497f77167a74589cf: Time spent bootstrapping tablet: real 0.014s user 0.001s sys 0.009s
I20260924 13:50:12.504438 13911 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJN9WRF/build/release/bin/kudu
/tmp/dist-test-taskJN9WRF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790257629041645-13911-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790257629041645-13911-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790257629041645-13911-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790257629041645-13911-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.149.194:45113
--local_ip_for_outbound_sockets=127.13.149.194
--tserver_master_addrs=127.13.149.254:36075
--webserver_port=42307
--webserver_interface=127.13.149.194
--keytab_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.194.keytab
--principal=kudu/127.13.149.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.149.212:45289
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790257629041645-13911-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790257629041645-13911-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790257629041645-13911-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260924 13:50:12.505435 28915 raft_consensus.cc:348] T 86dac4433c7d404aabd4a777b6c89ccb P 46b4fd024b654a5497f77167a74589cf [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8423e2366dc04ab8b01dc9d932e8ee75" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 45095 } } peers { permanent_uuid: "46b4fd024b654a5497f77167a74589cf" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 43429 } } peers { permanent_uuid: "666006d5e083435b9d59291a298b266c" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 45113 } }
I20260924 13:50:12.505666 28915 raft_consensus.cc:729] T 86dac4433c7d404aabd4a777b6c89ccb P 46b4fd024b654a5497f77167a74589cf [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 46b4fd024b654a5497f77167a74589cf, State: Initialized, Role: FOLLOWER
I20260924 13:50:12.505831 28915 consensus_queue.cc:260] T 86dac4433c7d404aabd4a777b6c89ccb P 46b4fd024b654a5497f77167a74589cf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8423e2366dc04ab8b01dc9d932e8ee75" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 45095 } } peers { permanent_uuid: "46b4fd024b654a5497f77167a74589cf" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 43429 } } peers { permanent_uuid: "666006d5e083435b9d59291a298b266c" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 45113 } }
I20260924 13:50:12.506088 28915 ts_tablet_manager.cc:1428] T 86dac4433c7d404aabd4a777b6c89ccb P 46b4fd024b654a5497f77167a74589cf: Time spent starting tablet: real 0.002s user 0.004s sys 0.001s
W20260924 13:50:12.555398 29025 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 13:50:12.498760 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:36075 (local address 127.13.149.193:59699)
0924 13:50:12.498965 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0924 13:50:12.499024 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0924 13:50:12.499676 (+ 652us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:50:12.555189 (+ 55513us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.149.254:36075: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":88,"threads_started":1}
W20260924 13:50:12.555665 29023 heartbeater.cc:644] Failed to heartbeat to 127.13.149.254:36075 (0 consecutive failures): Network error: Failed to ping master at 127.13.149.254:36075: Client connection negotiation failed: client connection to 127.13.149.254:36075: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260924 13:50:12.556046 29025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:50:12.555916 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:36075 (local address 127.13.149.193:57391)
0924 13:50:12.555973 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0924 13:50:12.556021 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.149.254:36075: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260924 13:50:12.556385 29025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:50:12.556295 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:36075 (local address 127.13.149.193:32853)
0924 13:50:12.556336 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0924 13:50:12.556364 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.149.254:36075: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260924 13:50:12.556499 29023 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260924 13:50:12.587121 29027 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 13:50:12.587340 29027 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 13:50:12.587364 29027 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 13:50:12.588469 29027 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 13:50:12.588501 29027 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 13:50:12.588526 29027 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.149.194
I20260924 13:50:12.589989 29027 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.149.212:45289
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790257629041645-13911-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790257629041645-13911-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.194.keytab
--principal=kudu/127.13.149.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.149.194:45113
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790257629041645-13911-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.149.194
--webserver_port=42307
--tserver_master_addrs=127.13.149.254:36075
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29027
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.149.194
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790257629041645-13911-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 12:29:37 UTC on ccycloud.quasar-iqzbxn.root.comops.site
build id 4670106
I20260924 13:50:12.590200 29027 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 13:50:12.590394 29027 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 13:50:12.595429 29034 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 13:50:12.595471 29036 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 13:50:12.595429 29033 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 13:50:12.595706 29027 server_base.cc:1034] running on GCE node
I20260924 13:50:12.595889 29027 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 13:50:12.596086 29027 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 13:50:12.597227 29027 hybrid_clock.cc:648] HybridClock initialized: now 1790257812597209 us; error 30 us; skew 500 ppm
Sep 24 13:50:12 dist-test-slave-sjl2 krb5kdc[27721](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257812, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.149.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 13:50:12.599743 29027 init.cc:377] Logged in from keytab as kudu/127.13.149.194@KRBTEST.COM (short username kudu)
I20260924 13:50:12.600232 29027 webserver.cc:460] Webserver started at http://127.13.149.194:42307/ using document root <none> and password file <none>
I20260924 13:50:12.600457 29027 fs_manager.cc:359] Metadata directory not provided
I20260924 13:50:12.600512 29027 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 13:50:12.602173 29027 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260924 13:50:12.602797 29043 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 13:50:12.602964 29027 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 13:50:12.603049 29027 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790257629041645-13911-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790257629041645-13911-0/minicluster-data/ts-1/wal
uuid: "666006d5e083435b9d59291a298b266c"
format_stamp: "Formatted at 2026-09-24 13:50:11 on dist-test-slave-sjl2"
server_key: "44b1ebca15f1cd56048d3c315cd6857a"
server_key_iv: "a09f710677d3ee077cb6294c3dec6b22"
server_key_version: "encryptionkey@0"
I20260924 13:50:12.603400 29027 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790257629041645-13911-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790257629041645-13911-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790257629041645-13911-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 13:50:12.614038 29027 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 13:50:12.616102 29027 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 13:50:12.616258 29027 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 13:50:12.616480 29027 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 13:50:12.616770 29027 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 13:50:12.616802 29027 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 13:50:12.616833 29027 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 13:50:12.616856 29027 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 13:50:12.622452 29027 rpc_server.cc:307] RPC server started. Bound to: 127.13.149.194:45113
I20260924 13:50:12.622498 29156 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.149.194:45113 every 8 connection(s)
I20260924 13:50:12.622856 29027 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790257629041645-13911-0/minicluster-data/ts-1/data/info.pb
I20260924 13:50:12.624007 29159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:50:12.623551 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:36075 (local address 127.13.149.194:53845)
0924 13:50:12.623768 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0924 13:50:12.623865 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.149.254:36075: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":63,"threads_started":1}
W20260924 13:50:12.624447 29157 heartbeater.cc:644] Failed to heartbeat to 127.13.149.254:36075 (0 consecutive failures): Network error: Failed to ping master at 127.13.149.254:36075: Client connection negotiation failed: client connection to 127.13.149.254:36075: connect: Connection refused (error 111)
I20260924 13:50:12.624738 29159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:50:12.624648 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:36075 (local address 127.13.149.194:47297)
0924 13:50:12.624690 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0924 13:50:12.624719 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.149.254:36075: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
I20260924 13:50:12.625140 29159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:50:12.625035 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:36075 (local address 127.13.149.194:52049)
0924 13:50:12.625062 (+ 27us) negotiation.cc:107] Waiting for socket to connect
0924 13:50:12.625082 (+ 20us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.149.254:36075: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":15}
W20260924 13:50:12.625257 29157 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260924 13:50:12.630198 13911 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJN9WRF/build/release/bin/kudu as pid 29027
I20260924 13:50:12.630450 13911 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJN9WRF/build/release/bin/kudu
/tmp/dist-test-taskJN9WRF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790257629041645-13911-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790257629041645-13911-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790257629041645-13911-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790257629041645-13911-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.149.195:45095
--local_ip_for_outbound_sockets=127.13.149.195
--tserver_master_addrs=127.13.149.254:36075
--webserver_port=41563
--webserver_interface=127.13.149.195
--keytab_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.195.keytab
--principal=kudu/127.13.149.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.149.212:45289
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790257629041645-13911-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790257629041645-13911-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790257629041645-13911-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 13:50:12.713289 29160 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 13:50:12.713522 29160 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 13:50:12.713550 29160 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 13:50:12.714910 29160 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 13:50:12.714969 29160 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 13:50:12.714999 29160 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.149.195
I20260924 13:50:12.716596 29160 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.149.212:45289
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790257629041645-13911-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790257629041645-13911-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.195.keytab
--principal=kudu/127.13.149.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.149.195:45095
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790257629041645-13911-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.149.195
--webserver_port=41563
--tserver_master_addrs=127.13.149.254:36075
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29160
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.149.195
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790257629041645-13911-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 12:29:37 UTC on ccycloud.quasar-iqzbxn.root.comops.site
build id 4670106
I20260924 13:50:12.716858 29160 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 13:50:12.717136 29160 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 13:50:12.722180 29166 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 13:50:12.722209 29165 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 13:50:12.722268 29168 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 13:50:12.722612 29160 server_base.cc:1034] running on GCE node
I20260924 13:50:12.722841 29160 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 13:50:12.723100 29160 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 13:50:12.724238 29160 hybrid_clock.cc:648] HybridClock initialized: now 1790257812724222 us; error 34 us; skew 500 ppm
Sep 24 13:50:12 dist-test-slave-sjl2 krb5kdc[27721](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257812, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.149.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 13:50:12.726822 29160 init.cc:377] Logged in from keytab as kudu/127.13.149.195@KRBTEST.COM (short username kudu)
I20260924 13:50:12.727269 29160 webserver.cc:460] Webserver started at http://127.13.149.195:41563/ using document root <none> and password file <none>
I20260924 13:50:12.727476 29160 fs_manager.cc:359] Metadata directory not provided
I20260924 13:50:12.727520 29160 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 13:50:12.728907 29160 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260924 13:50:12.729528 29175 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 13:50:12.729671 29160 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260924 13:50:12.729810 29160 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790257629041645-13911-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790257629041645-13911-0/minicluster-data/ts-2/wal
uuid: "8423e2366dc04ab8b01dc9d932e8ee75"
format_stamp: "Formatted at 2026-09-24 13:50:11 on dist-test-slave-sjl2"
server_key: "83f8cad45ec6d96e29788656175a6647"
server_key_iv: "6b65e4d52a42cc1cdac48a416fe21a38"
server_key_version: "encryptionkey@0"
I20260924 13:50:12.730193 29160 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790257629041645-13911-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790257629041645-13911-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790257629041645-13911-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 13:50:12.751989 29160 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 13:50:12.754297 29160 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 13:50:12.754431 29160 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 13:50:12.754698 29160 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 13:50:12.755028 29160 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 13:50:12.755071 29160 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 13:50:12.755101 29160 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 13:50:12.755118 29160 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 13:50:12.760874 29160 rpc_server.cc:307] RPC server started. Bound to: 127.13.149.195:45095
I20260924 13:50:12.760962 29288 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.149.195:45095 every 8 connection(s)
I20260924 13:50:12.761309 29160 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790257629041645-13911-0/minicluster-data/ts-2/data/info.pb
I20260924 13:50:12.762421 29291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:50:12.762013 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:36075 (local address 127.13.149.195:59203)
0924 13:50:12.762169 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0924 13:50:12.762276 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.149.254:36075: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":73,"threads_started":1}
W20260924 13:50:12.762907 29289 heartbeater.cc:644] Failed to heartbeat to 127.13.149.254:36075 (0 consecutive failures): Network error: Failed to ping master at 127.13.149.254:36075: Client connection negotiation failed: client connection to 127.13.149.254:36075: connect: Connection refused (error 111)
I20260924 13:50:12.763221 29291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:50:12.763123 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:36075 (local address 127.13.149.195:38987)
0924 13:50:12.763167 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0924 13:50:12.763199 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.149.254:36075: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
I20260924 13:50:12.763459 29291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:50:12.763397 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:36075 (local address 127.13.149.195:47139)
0924 13:50:12.763428 (+ 31us) negotiation.cc:107] Waiting for socket to connect
0924 13:50:12.763447 (+ 19us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.149.254:36075: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
W20260924 13:50:12.763545 29289 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260924 13:50:12.765618 13911 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJN9WRF/build/release/bin/kudu as pid 29160
I20260924 13:50:13.557250 29293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:50:13.556994 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:36075 (local address 127.13.149.193:57875)
0924 13:50:13.557132 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0924 13:50:13.557177 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.149.254:36075: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":64,"threads_started":1}
I20260924 13:50:13.625703 29294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:50:13.625507 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:36075 (local address 127.13.149.194:53139)
0924 13:50:13.625613 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0924 13:50:13.625648 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.149.254:36075: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":91,"thread_start_us":51,"threads_started":1}
I20260924 13:50:13.764113 29295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:50:13.763882 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:36075 (local address 127.13.149.195:53761)
0924 13:50:13.764019 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0924 13:50:13.764058 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.149.254:36075: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":73,"threads_started":1}
I20260924 13:50:13.822433 29296 raft_consensus.cc:482] T 86dac4433c7d404aabd4a777b6c89ccb P 46b4fd024b654a5497f77167a74589cf [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 13:50:13.822577 29296 raft_consensus.cc:504] T 86dac4433c7d404aabd4a777b6c89ccb P 46b4fd024b654a5497f77167a74589cf [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8423e2366dc04ab8b01dc9d932e8ee75" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 45095 } } peers { permanent_uuid: "46b4fd024b654a5497f77167a74589cf" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 43429 } } peers { permanent_uuid: "666006d5e083435b9d59291a298b266c" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 45113 } }
I20260924 13:50:13.822923 29296 leader_election.cc:288] T 86dac4433c7d404aabd4a777b6c89ccb P 46b4fd024b654a5497f77167a74589cf [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8423e2366dc04ab8b01dc9d932e8ee75 (127.13.149.195:45095), 666006d5e083435b9d59291a298b266c (127.13.149.194:45113)
Sep 24 13:50:13 dist-test-slave-sjl2 krb5kdc[27721](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257812, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.149.193@KRBTEST.COM for kudu/127.13.149.195@KRBTEST.COM
Sep 24 13:50:13 dist-test-slave-sjl2 krb5kdc[27721](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257812, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.149.193@KRBTEST.COM for kudu/127.13.149.194@KRBTEST.COM
I20260924 13:50:13.832902 29299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:50:13.823203 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.149.193:45875 (local address 127.13.149.195:45095)
0924 13:50:13.823396 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0924 13:50:13.823404 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0924 13:50:13.823427 (+ 23us) server_negotiation.cc:407] Connection header received
0924 13:50:13.823850 (+ 423us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:50:13.823852 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:50:13.823891 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:50:13.824013 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:50:13.824724 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:50:13.825754 (+ 1030us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:50:13.826682 (+ 928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:50:13.827071 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:50:13.827167 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:50:13.829353 (+ 2186us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:50:13.829357 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:50:13.829357 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 13:50:13.829367 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:50:13.831035 (+ 1668us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:50:13.831517 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:50:13.831521 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:50:13.831522 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:50:13.831633 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:50:13.831921 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:50:13.831922 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:50:13.831922 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:50:13.832248 (+ 326us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:50:13.832310 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0924 13:50:13.832633 (+ 323us) server_negotiation.cc:299] Negotiation successful
0924 13:50:13.832788 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":166,"thread_start_us":95,"threads_started":1}
I20260924 13:50:13.832942 29298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:50:13.823197 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.149.193:37035 (local address 127.13.149.194:45113)
0924 13:50:13.823391 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0924 13:50:13.823402 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0924 13:50:13.823430 (+ 28us) server_negotiation.cc:407] Connection header received
0924 13:50:13.823854 (+ 424us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:50:13.823856 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:50:13.823891 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:50:13.824013 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:50:13.824718 (+ 705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:50:13.825707 (+ 989us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:50:13.826682 (+ 975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:50:13.827071 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:50:13.827167 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:50:13.829690 (+ 2523us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:50:13.829694 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:50:13.829694 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 13:50:13.829705 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:50:13.831319 (+ 1614us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:50:13.831792 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:50:13.831795 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:50:13.831795 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:50:13.831897 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:50:13.832095 (+ 198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:50:13.832097 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:50:13.832097 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:50:13.832438 (+ 341us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:50:13.832517 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0924 13:50:13.832664 (+ 147us) server_negotiation.cc:299] Negotiation successful
0924 13:50:13.832794 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":164,"thread_start_us":94,"threads_started":1}
I20260924 13:50:13.832901 29293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:50:13.823096 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.195:45095 (local address 127.13.149.193:45875)
0924 13:50:13.823213 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0924 13:50:13.823227 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0924 13:50:13.823297 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:50:13.824058 (+ 761us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:50:13.824060 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:50:13.824421 (+ 361us) client_negotiation.cc:189] Negotiated authn=SASL
0924 13:50:13.824609 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:50:13.824614 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:50:13.825840 (+ 1226us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:50:13.825842 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:50:13.826605 (+ 763us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:50:13.826607 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:50:13.827151 (+ 544us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:50:13.827152 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:50:13.827260 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:50:13.827817 (+ 557us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 13:50:13.827821 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0924 13:50:13.829252 (+ 1431us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 13:50:13.831125 (+ 1873us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 13:50:13.831135 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 13:50:13.831137 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 13:50:13.831417 (+ 280us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 13:50:13.831715 (+ 298us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 13:50:13.831716 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 13:50:13.831716 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 13:50:13.831851 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 13:50:13.832326 (+ 475us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 13:50:13.832332 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 13:50:13.832466 (+ 134us) client_negotiation.cc:769] Sending connection context
0924 13:50:13.832570 (+ 104us) client_negotiation.cc:240] Negotiation successful
0924 13:50:13.832790 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":93,"mutex_wait_us":38}
I20260924 13:50:13.833587 29297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:50:13.823096 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.194:45113 (local address 127.13.149.193:37035)
0924 13:50:13.823239 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0924 13:50:13.823250 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0924 13:50:13.823308 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:50:13.824059 (+ 751us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:50:13.824061 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:50:13.824423 (+ 362us) client_negotiation.cc:189] Negotiated authn=SASL
0924 13:50:13.824609 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:50:13.824614 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:50:13.825797 (+ 1183us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:50:13.825798 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:50:13.826606 (+ 808us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:50:13.826609 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:50:13.827148 (+ 539us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:50:13.827149 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:50:13.827260 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:50:13.827767 (+ 507us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 13:50:13.827774 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0924 13:50:13.829597 (+ 1823us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 13:50:13.831496 (+ 1899us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 13:50:13.831497 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 13:50:13.831498 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 13:50:13.831713 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 13:50:13.831955 (+ 242us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 13:50:13.831956 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 13:50:13.831956 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 13:50:13.832016 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 13:50:13.832496 (+ 480us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 13:50:13.832497 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 13:50:13.832534 (+ 37us) client_negotiation.cc:769] Sending connection context
0924 13:50:13.832599 (+ 65us) client_negotiation.cc:240] Negotiation successful
0924 13:50:13.832867 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"mutex_wait_us":214,"thread_start_us":46,"threads_started":1}
W20260924 13:50:13.833838 28912 leader_election.cc:341] T 86dac4433c7d404aabd4a777b6c89ccb P 46b4fd024b654a5497f77167a74589cf [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8423e2366dc04ab8b01dc9d932e8ee75 (127.13.149.195:45095): Not found: Tablet not found: 86dac4433c7d404aabd4a777b6c89ccb
W20260924 13:50:13.834201 28910 leader_election.cc:341] T 86dac4433c7d404aabd4a777b6c89ccb P 46b4fd024b654a5497f77167a74589cf [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 666006d5e083435b9d59291a298b266c (127.13.149.194:45113): Not found: Tablet not found: 86dac4433c7d404aabd4a777b6c89ccb
I20260924 13:50:13.834262 28910 leader_election.cc:302] T 86dac4433c7d404aabd4a777b6c89ccb P 46b4fd024b654a5497f77167a74589cf [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46b4fd024b654a5497f77167a74589cf; no voters: 666006d5e083435b9d59291a298b266c, 8423e2366dc04ab8b01dc9d932e8ee75
I20260924 13:50:13.834355 29296 raft_consensus.cc:2726] T 86dac4433c7d404aabd4a777b6c89ccb P 46b4fd024b654a5497f77167a74589cf [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260924 13:50:14.557996 29300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:50:14.557745 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:36075 (local address 127.13.149.193:38519)
0924 13:50:14.557905 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0924 13:50:14.557948 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.149.254:36075: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":69,"threads_started":1}
I20260924 13:50:14.626335 29301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:50:14.626121 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:36075 (local address 127.13.149.194:48633)
0924 13:50:14.626250 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0924 13:50:14.626293 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.149.254:36075: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":58,"threads_started":1}
I20260924 13:50:14.764796 29302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:50:14.764590 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:36075 (local address 127.13.149.195:60703)
0924 13:50:14.764704 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0924 13:50:14.764738 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.149.254:36075: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":64,"threads_started":1}
I20260924 13:50:15.558782 29303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:50:15.558520 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:36075 (local address 127.13.149.193:32877)
0924 13:50:15.558686 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0924 13:50:15.558733 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.149.254:36075: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":73,"threads_started":1}
I20260924 13:50:15.626979 29304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:50:15.626756 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:36075 (local address 127.13.149.194:60499)
0924 13:50:15.626889 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0924 13:50:15.626934 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.149.254:36075: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":61,"threads_started":1}
I20260924 13:50:15.765568 29305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:50:15.765311 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:36075 (local address 127.13.149.195:53455)
0924 13:50:15.765461 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0924 13:50:15.765513 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.149.254:36075: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":60,"threads_started":1}
I20260924 13:50:16.082612 29306 raft_consensus.cc:482] T 86dac4433c7d404aabd4a777b6c89ccb P 46b4fd024b654a5497f77167a74589cf [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 13:50:16.082703 29306 raft_consensus.cc:504] T 86dac4433c7d404aabd4a777b6c89ccb P 46b4fd024b654a5497f77167a74589cf [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8423e2366dc04ab8b01dc9d932e8ee75" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 45095 } } peers { permanent_uuid: "46b4fd024b654a5497f77167a74589cf" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 43429 } } peers { permanent_uuid: "666006d5e083435b9d59291a298b266c" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 45113 } }
I20260924 13:50:16.082880 29306 leader_election.cc:288] T 86dac4433c7d404aabd4a777b6c89ccb P 46b4fd024b654a5497f77167a74589cf [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8423e2366dc04ab8b01dc9d932e8ee75 (127.13.149.195:45095), 666006d5e083435b9d59291a298b266c (127.13.149.194:45113)
W20260924 13:50:16.083225 28910 leader_election.cc:341] T 86dac4433c7d404aabd4a777b6c89ccb P 46b4fd024b654a5497f77167a74589cf [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 666006d5e083435b9d59291a298b266c (127.13.149.194:45113): Not found: Tablet not found: 86dac4433c7d404aabd4a777b6c89ccb
W20260924 13:50:16.083320 28912 leader_election.cc:341] T 86dac4433c7d404aabd4a777b6c89ccb P 46b4fd024b654a5497f77167a74589cf [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8423e2366dc04ab8b01dc9d932e8ee75 (127.13.149.195:45095): Not found: Tablet not found: 86dac4433c7d404aabd4a777b6c89ccb
I20260924 13:50:16.083380 28912 leader_election.cc:302] T 86dac4433c7d404aabd4a777b6c89ccb P 46b4fd024b654a5497f77167a74589cf [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46b4fd024b654a5497f77167a74589cf; no voters: 666006d5e083435b9d59291a298b266c, 8423e2366dc04ab8b01dc9d932e8ee75
I20260924 13:50:16.083487 29306 raft_consensus.cc:2726] T 86dac4433c7d404aabd4a777b6c89ccb P 46b4fd024b654a5497f77167a74589cf [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260924 13:50:16.559561 29307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:50:16.559333 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:36075 (local address 127.13.149.193:41907)
0924 13:50:16.559474 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0924 13:50:16.559511 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.149.254:36075: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":66,"threads_started":1}
I20260924 13:50:16.627696 29308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:50:16.627481 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:36075 (local address 127.13.149.194:48793)
0924 13:50:16.627617 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0924 13:50:16.627654 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.149.254:36075: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":60,"threads_started":1}
I20260924 13:50:16.766307 29309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:50:16.766092 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:36075 (local address 127.13.149.195:59171)
0924 13:50:16.766220 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0924 13:50:16.766261 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.149.254:36075: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":70,"threads_started":1}
I20260924 13:50:17.560353 29310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:50:17.560144 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:36075 (local address 127.13.149.193:34227)
0924 13:50:17.560264 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0924 13:50:17.560308 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.149.254:36075: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":61,"threads_started":1}
I20260924 13:50:17.628427 29311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:50:17.628193 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:36075 (local address 127.13.149.194:48583)
0924 13:50:17.628337 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0924 13:50:17.628376 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.149.254:36075: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":62,"threads_started":1}
I20260924 13:50:17.767015 29312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:50:17.766807 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:36075 (local address 127.13.149.195:39245)
0924 13:50:17.766925 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0924 13:50:17.766968 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.149.254:36075: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":56,"threads_started":1}
I20260924 13:50:17.938635 29313 raft_consensus.cc:482] T 86dac4433c7d404aabd4a777b6c89ccb P 46b4fd024b654a5497f77167a74589cf [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 13:50:17.938752 29313 raft_consensus.cc:504] T 86dac4433c7d404aabd4a777b6c89ccb P 46b4fd024b654a5497f77167a74589cf [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8423e2366dc04ab8b01dc9d932e8ee75" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 45095 } } peers { permanent_uuid: "46b4fd024b654a5497f77167a74589cf" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 43429 } } peers { permanent_uuid: "666006d5e083435b9d59291a298b266c" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 45113 } }
I20260924 13:50:17.938932 29313 leader_election.cc:288] T 86dac4433c7d404aabd4a777b6c89ccb P 46b4fd024b654a5497f77167a74589cf [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8423e2366dc04ab8b01dc9d932e8ee75 (127.13.149.195:45095), 666006d5e083435b9d59291a298b266c (127.13.149.194:45113)
W20260924 13:50:17.939389 28912 leader_election.cc:341] T 86dac4433c7d404aabd4a777b6c89ccb P 46b4fd024b654a5497f77167a74589cf [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8423e2366dc04ab8b01dc9d932e8ee75 (127.13.149.195:45095): Not found: Tablet not found: 86dac4433c7d404aabd4a777b6c89ccb
W20260924 13:50:17.939488 28910 leader_election.cc:341] T 86dac4433c7d404aabd4a777b6c89ccb P 46b4fd024b654a5497f77167a74589cf [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 666006d5e083435b9d59291a298b266c (127.13.149.194:45113): Not found: Tablet not found: 86dac4433c7d404aabd4a777b6c89ccb
I20260924 13:50:17.939558 28910 leader_election.cc:302] T 86dac4433c7d404aabd4a777b6c89ccb P 46b4fd024b654a5497f77167a74589cf [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46b4fd024b654a5497f77167a74589cf; no voters: 666006d5e083435b9d59291a298b266c, 8423e2366dc04ab8b01dc9d932e8ee75
I20260924 13:50:17.939638 29313 raft_consensus.cc:2726] T 86dac4433c7d404aabd4a777b6c89ccb P 46b4fd024b654a5497f77167a74589cf [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260924 13:50:18.561096 29314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:50:18.560830 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:36075 (local address 127.13.149.193:42063)
0924 13:50:18.561002 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0924 13:50:18.561047 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.149.254:36075: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":107,"threads_started":1}
I20260924 13:50:18.629114 29315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:50:18.628874 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:36075 (local address 127.13.149.194:43709)
0924 13:50:18.629023 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0924 13:50:18.629063 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.149.254:36075: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":96,"threads_started":1}
I20260924 13:50:18.767637 29316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:50:18.767431 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:36075 (local address 127.13.149.195:53877)
0924 13:50:18.767547 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0924 13:50:18.767591 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.149.254:36075: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":58,"threads_started":1}
I20260924 13:50:19.561874 29317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:50:19.561648 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:36075 (local address 127.13.149.193:56115)
0924 13:50:19.561772 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0924 13:50:19.561818 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.149.254:36075: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":63,"threads_started":1}
I20260924 13:50:19.629760 29318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:50:19.629506 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:36075 (local address 127.13.149.194:41283)
0924 13:50:19.629670 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0924 13:50:19.629712 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.149.254:36075: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":74,"threads_started":1}
I20260924 13:50:19.768352 29319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:50:19.768100 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:36075 (local address 127.13.149.195:39149)
0924 13:50:19.768256 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0924 13:50:19.768304 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.149.254:36075: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":62,"threads_started":1}
I20260924 13:50:19.998159 29320 raft_consensus.cc:482] T 86dac4433c7d404aabd4a777b6c89ccb P 46b4fd024b654a5497f77167a74589cf [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 13:50:19.998260 29320 raft_consensus.cc:504] T 86dac4433c7d404aabd4a777b6c89ccb P 46b4fd024b654a5497f77167a74589cf [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8423e2366dc04ab8b01dc9d932e8ee75" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 45095 } } peers { permanent_uuid: "46b4fd024b654a5497f77167a74589cf" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 43429 } } peers { permanent_uuid: "666006d5e083435b9d59291a298b266c" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 45113 } }
I20260924 13:50:19.998423 29320 leader_election.cc:288] T 86dac4433c7d404aabd4a777b6c89ccb P 46b4fd024b654a5497f77167a74589cf [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8423e2366dc04ab8b01dc9d932e8ee75 (127.13.149.195:45095), 666006d5e083435b9d59291a298b266c (127.13.149.194:45113)
W20260924 13:50:19.998798 28910 leader_election.cc:341] T 86dac4433c7d404aabd4a777b6c89ccb P 46b4fd024b654a5497f77167a74589cf [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 666006d5e083435b9d59291a298b266c (127.13.149.194:45113): Not found: Tablet not found: 86dac4433c7d404aabd4a777b6c89ccb
W20260924 13:50:19.998947 28912 leader_election.cc:341] T 86dac4433c7d404aabd4a777b6c89ccb P 46b4fd024b654a5497f77167a74589cf [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8423e2366dc04ab8b01dc9d932e8ee75 (127.13.149.195:45095): Not found: Tablet not found: 86dac4433c7d404aabd4a777b6c89ccb
I20260924 13:50:19.999020 28912 leader_election.cc:302] T 86dac4433c7d404aabd4a777b6c89ccb P 46b4fd024b654a5497f77167a74589cf [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46b4fd024b654a5497f77167a74589cf; no voters: 666006d5e083435b9d59291a298b266c, 8423e2366dc04ab8b01dc9d932e8ee75
I20260924 13:50:19.999100 29320 raft_consensus.cc:2726] T 86dac4433c7d404aabd4a777b6c89ccb P 46b4fd024b654a5497f77167a74589cf [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260924 13:50:20.562603 29321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:50:20.562372 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:36075 (local address 127.13.149.193:42399)
0924 13:50:20.562509 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0924 13:50:20.562554 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.149.254:36075: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":79,"threads_started":1}
I20260924 13:50:20.630554 29322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:50:20.630324 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:36075 (local address 127.13.149.194:35695)
0924 13:50:20.630463 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0924 13:50:20.630507 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.149.254:36075: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":63,"threads_started":1}
I20260924 13:50:20.769088 29323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:50:20.768829 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:36075 (local address 127.13.149.195:54255)
0924 13:50:20.768996 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0924 13:50:20.769039 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.149.254:36075: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":58,"threads_started":1}
I20260924 13:50:21.563352 29324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:50:21.563116 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:36075 (local address 127.13.149.193:49969)
0924 13:50:21.563257 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0924 13:50:21.563304 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.149.254:36075: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":63,"threads_started":1}
I20260924 13:50:21.631259 29325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:50:21.631035 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:36075 (local address 127.13.149.194:42901)
0924 13:50:21.631172 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0924 13:50:21.631211 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.149.254:36075: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":61,"threads_started":1}
I20260924 13:50:21.769817 29326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:50:21.769576 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:36075 (local address 127.13.149.195:55165)
0924 13:50:21.769726 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0924 13:50:21.769767 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.149.254:36075: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":80,"threads_started":1}
I20260924 13:50:22.564082 29327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:50:22.563848 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:36075 (local address 127.13.149.193:33697)
0924 13:50:22.563989 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0924 13:50:22.564034 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.149.254:36075: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":61,"threads_started":1}
I20260924 13:50:22.631906 29328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:50:22.631680 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:36075 (local address 127.13.149.194:58545)
0924 13:50:22.631825 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0924 13:50:22.631859 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.149.254:36075: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":65,"threads_started":1}
I20260924 13:50:22.770514 29329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:50:22.770280 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:36075 (local address 127.13.149.195:60023)
0924 13:50:22.770426 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0924 13:50:22.770468 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.149.254:36075: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":71,"threads_started":1}
I20260924 13:50:23.564807 29330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:50:23.564568 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:36075 (local address 127.13.149.193:60783)
0924 13:50:23.564709 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0924 13:50:23.564759 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.149.254:36075: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":63,"threads_started":1}
I20260924 13:50:23.632494 29331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:50:23.632291 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:36075 (local address 127.13.149.194:53047)
0924 13:50:23.632414 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0924 13:50:23.632448 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.149.254:36075: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":63,"threads_started":1}
I20260924 13:50:23.771224 29332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:50:23.770981 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:36075 (local address 127.13.149.195:34647)
0924 13:50:23.771133 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0924 13:50:23.771176 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.149.254:36075: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":74,"threads_started":1}
I20260924 13:50:24.565630 29333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:50:24.565366 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:36075 (local address 127.13.149.193:59217)
0924 13:50:24.565520 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0924 13:50:24.565570 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.149.254:36075: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":65,"threads_started":1}
I20260924 13:50:24.633265 29334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:50:24.633002 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:36075 (local address 127.13.149.194:47305)
0924 13:50:24.633150 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0924 13:50:24.633204 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.149.254:36075: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":62,"threads_started":1}
I20260924 13:50:24.771849 29335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:50:24.771611 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:36075 (local address 127.13.149.195:56717)
0924 13:50:24.771732 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0924 13:50:24.771769 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.149.254:36075: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":65,"threads_started":1}
I20260924 13:50:24.774753 29336 raft_consensus.cc:482] T 86dac4433c7d404aabd4a777b6c89ccb P 46b4fd024b654a5497f77167a74589cf [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 13:50:24.774837 29336 raft_consensus.cc:504] T 86dac4433c7d404aabd4a777b6c89ccb P 46b4fd024b654a5497f77167a74589cf [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8423e2366dc04ab8b01dc9d932e8ee75" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 45095 } } peers { permanent_uuid: "46b4fd024b654a5497f77167a74589cf" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 43429 } } peers { permanent_uuid: "666006d5e083435b9d59291a298b266c" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 45113 } }
I20260924 13:50:24.774968 29336 leader_election.cc:288] T 86dac4433c7d404aabd4a777b6c89ccb P 46b4fd024b654a5497f77167a74589cf [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8423e2366dc04ab8b01dc9d932e8ee75 (127.13.149.195:45095), 666006d5e083435b9d59291a298b266c (127.13.149.194:45113)
W20260924 13:50:24.775301 28912 leader_election.cc:341] T 86dac4433c7d404aabd4a777b6c89ccb P 46b4fd024b654a5497f77167a74589cf [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8423e2366dc04ab8b01dc9d932e8ee75 (127.13.149.195:45095): Not found: Tablet not found: 86dac4433c7d404aabd4a777b6c89ccb
W20260924 13:50:24.775386 28910 leader_election.cc:341] T 86dac4433c7d404aabd4a777b6c89ccb P 46b4fd024b654a5497f77167a74589cf [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 666006d5e083435b9d59291a298b266c (127.13.149.194:45113): Not found: Tablet not found: 86dac4433c7d404aabd4a777b6c89ccb
I20260924 13:50:24.775429 28910 leader_election.cc:302] T 86dac4433c7d404aabd4a777b6c89ccb P 46b4fd024b654a5497f77167a74589cf [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46b4fd024b654a5497f77167a74589cf; no voters: 666006d5e083435b9d59291a298b266c, 8423e2366dc04ab8b01dc9d932e8ee75
I20260924 13:50:24.775507 29336 raft_consensus.cc:2726] T 86dac4433c7d404aabd4a777b6c89ccb P 46b4fd024b654a5497f77167a74589cf [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260924 13:50:25.566279 29337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:50:25.566074 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:36075 (local address 127.13.149.193:59405)
0924 13:50:25.566193 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0924 13:50:25.566233 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.149.254:36075: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":62,"threads_started":1}
I20260924 13:50:25.634024 29338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:50:25.633781 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:36075 (local address 127.13.149.194:53665)
0924 13:50:25.633907 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0924 13:50:25.633952 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.149.254:36075: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":61,"threads_started":1}
I20260924 13:50:25.772531 29339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:50:25.772283 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:36075 (local address 127.13.149.195:57243)
0924 13:50:25.772439 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0924 13:50:25.772484 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.149.254:36075: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":79,"threads_started":1}
I20260924 13:50:26.566978 29340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:50:26.566733 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:36075 (local address 127.13.149.193:48923)
0924 13:50:26.566892 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0924 13:50:26.566931 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.149.254:36075: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":73,"threads_started":1}
I20260924 13:50:26.634673 29341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:50:26.634442 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:36075 (local address 127.13.149.194:50739)
0924 13:50:26.634575 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0924 13:50:26.634623 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.149.254:36075: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":58,"threads_started":1}
I20260924 13:50:26.773223 29342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:50:26.773013 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:36075 (local address 127.13.149.195:42627)
0924 13:50:26.773136 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0924 13:50:26.773177 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.149.254:36075: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":70,"threads_started":1}
I20260924 13:50:27.567767 29343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:50:27.567564 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:36075 (local address 127.13.149.193:33203)
0924 13:50:27.567687 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0924 13:50:27.567719 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.149.254:36075: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":62,"threads_started":1}
I20260924 13:50:27.635500 29344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:50:27.635251 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:36075 (local address 127.13.149.194:60437)
0924 13:50:27.635399 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0924 13:50:27.635450 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.149.254:36075: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":62,"threads_started":1}
I20260924 13:50:27.766286 13911 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJN9WRF/build/release/bin/kudu with pid 28864
I20260924 13:50:27.770761 13911 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJN9WRF/build/release/bin/kudu with pid 29027
I20260924 13:50:27.773880 29345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:50:27.773690 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:36075 (local address 127.13.149.195:45469)
0924 13:50:27.773807 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0924 13:50:27.773838 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.149.254:36075: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":55,"threads_started":1}
I20260924 13:50:27.774684 13911 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJN9WRF/build/release/bin/kudu with pid 29160
2026-09-24T13:50:27Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (18100 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790257629041645-13911-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 13:50:27 dist-test-slave-sjl2 krb5kdc[29348](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 13:50:27 dist-test-slave-sjl2 krb5kdc[29348](info): set up 2 sockets
Sep 24 13:50:27 dist-test-slave-sjl2 krb5kdc[29348](info): commencing operation
krb5kdc: starting...
W20260924 13:50:28.733853 13911 mini_kdc.cc:121] Time spent starting KDC: real 0.940s user 0.003s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790257629041645-13911-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790257629041645-13911-0/krb5kdc/test-user.keytab.
Sep 24 13:50:28 dist-test-slave-sjl2 krb5kdc[29348](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257828, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T13:50:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T13:50:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.149.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.149.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.149.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.254.keytab.
Entry for principal kudu/127.13.149.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.254.keytab.
I20260924 13:50:28.851838 13911 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJN9WRF/build/release/bin/kudu
/tmp/dist-test-taskJN9WRF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790257629041645-13911-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790257629041645-13911-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790257629041645-13911-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790257629041645-13911-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.149.254:38197
--webserver_interface=127.13.149.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.254.keytab
--principal=kudu/127.13.149.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.149.212:44125
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.149.254:38197
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790257629041645-13911-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790257629041645-13911-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790257629041645-13911-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 13:50:28.931319 29361 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 13:50:28.931536 29361 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 13:50:28.931567 29361 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 13:50:28.932716 29361 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260924 13:50:28.932754 29361 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260924 13:50:28.932771 29361 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260924 13:50:28.932785 29361 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 13:50:28.932802 29361 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 13:50:28.932828 29361 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 13:50:28.932852 29361 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 13:50:28.932869 29361 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 13:50:28.934396 29361 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.149.212:44125
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790257629041645-13911-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790257629041645-13911-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.13.149.254:38197
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.254.keytab
--principal=kudu/127.13.149.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.149.254:38197
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790257629041645-13911-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.149.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29361
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790257629041645-13911-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 12:29:37 UTC on ccycloud.quasar-iqzbxn.root.comops.site
build id 4670106
I20260924 13:50:28.934644 29361 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 13:50:28.934860 29361 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 13:50:28.940061 29366 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 13:50:28.940193 29367 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 13:50:28.940145 29361 server_base.cc:1034] running on GCE node
W20260924 13:50:28.940155 29369 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 13:50:28.940521 29361 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 13:50:28.940820 29361 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 13:50:28.941969 29361 hybrid_clock.cc:648] HybridClock initialized: now 1790257828941946 us; error 34 us; skew 500 ppm
Sep 24 13:50:28 dist-test-slave-sjl2 krb5kdc[29348](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257828, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.149.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 13:50:28.944252 29361 init.cc:377] Logged in from keytab as kudu/127.13.149.254@KRBTEST.COM (short username kudu)
I20260924 13:50:28.944722 29361 webserver.cc:460] Webserver started at http://127.13.149.254:34079/ using document root <none> and password file <none>
I20260924 13:50:28.945060 29361 fs_manager.cc:359] Metadata directory not provided
I20260924 13:50:28.945122 29361 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 13:50:28.945281 29361 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 13:50:28.946338 29361 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790257629041645-13911-0/minicluster-data/master-0/data/instance:
uuid: "5d045e5a8ede44cba24125f78123692f"
format_stamp: "Formatted at 2026-09-24 13:50:28 on dist-test-slave-sjl2"
server_key: "cb869e500d58c01ee6af39032eb28186"
server_key_iv: "3152d23837859a2898ff482ee43ee262"
server_key_version: "encryptionkey@0"
I20260924 13:50:28.946659 29361 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790257629041645-13911-0/minicluster-data/master-0/wal/instance:
uuid: "5d045e5a8ede44cba24125f78123692f"
format_stamp: "Formatted at 2026-09-24 13:50:28 on dist-test-slave-sjl2"
server_key: "cb869e500d58c01ee6af39032eb28186"
server_key_iv: "3152d23837859a2898ff482ee43ee262"
server_key_version: "encryptionkey@0"
I20260924 13:50:28.947947 29361 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260924 13:50:28.948657 29376 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 13:50:28.948833 29361 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 13:50:28.948935 29361 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790257629041645-13911-0/minicluster-data/master-0/data,/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790257629041645-13911-0/minicluster-data/master-0/wal
uuid: "5d045e5a8ede44cba24125f78123692f"
format_stamp: "Formatted at 2026-09-24 13:50:28 on dist-test-slave-sjl2"
server_key: "cb869e500d58c01ee6af39032eb28186"
server_key_iv: "3152d23837859a2898ff482ee43ee262"
server_key_version: "encryptionkey@0"
I20260924 13:50:28.949005 29361 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790257629041645-13911-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790257629041645-13911-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790257629041645-13911-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 13:50:28.960955 29361 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 13:50:28.963155 29361 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 13:50:28.963297 29361 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 13:50:28.966989 29361 rpc_server.cc:307] RPC server started. Bound to: 127.13.149.254:38197
I20260924 13:50:28.967041 29428 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.149.254:38197 every 8 connection(s)
I20260924 13:50:28.967366 29361 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790257629041645-13911-0/minicluster-data/master-0/data/info.pb
I20260924 13:50:28.968215 29429 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 13:50:28.970532 29429 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5d045e5a8ede44cba24125f78123692f: Bootstrap starting.
I20260924 13:50:28.971064 29429 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5d045e5a8ede44cba24125f78123692f: Neither blocks nor log segments found. Creating new log.
I20260924 13:50:28.971303 29429 log.cc:824] T 00000000000000000000000000000000 P 5d045e5a8ede44cba24125f78123692f: Log is configured to *not* fsync() on all Append() calls
I20260924 13:50:28.971885 29429 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5d045e5a8ede44cba24125f78123692f: No bootstrap required, opened a new log
I20260924 13:50:28.973476 29429 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5d045e5a8ede44cba24125f78123692f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d045e5a8ede44cba24125f78123692f" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 38197 } }
I20260924 13:50:28.973614 29429 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5d045e5a8ede44cba24125f78123692f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 13:50:28.973636 29429 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5d045e5a8ede44cba24125f78123692f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5d045e5a8ede44cba24125f78123692f, State: Initialized, Role: FOLLOWER
I20260924 13:50:28.973749 29429 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5d045e5a8ede44cba24125f78123692f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d045e5a8ede44cba24125f78123692f" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 38197 } }
I20260924 13:50:28.973824 29429 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5d045e5a8ede44cba24125f78123692f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 13:50:28.973852 29429 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5d045e5a8ede44cba24125f78123692f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 13:50:28.973884 29429 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5d045e5a8ede44cba24125f78123692f [term 0 FOLLOWER]: Advancing to term 1
I20260924 13:50:28.974360 29429 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5d045e5a8ede44cba24125f78123692f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d045e5a8ede44cba24125f78123692f" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 38197 } }
I20260924 13:50:28.974442 29429 leader_election.cc:302] T 00000000000000000000000000000000 P 5d045e5a8ede44cba24125f78123692f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5d045e5a8ede44cba24125f78123692f; no voters:
I20260924 13:50:28.974565 29429 leader_election.cc:288] T 00000000000000000000000000000000 P 5d045e5a8ede44cba24125f78123692f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 13:50:28.974614 29432 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5d045e5a8ede44cba24125f78123692f [term 1 FOLLOWER]: Leader election won for term 1
I20260924 13:50:28.974776 29429 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5d045e5a8ede44cba24125f78123692f [sys.catalog]: configured and running, proceeding with master startup.
I20260924 13:50:28.974804 29432 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5d045e5a8ede44cba24125f78123692f [term 1 LEADER]: Becoming Leader. State: Replica: 5d045e5a8ede44cba24125f78123692f, State: Running, Role: LEADER
I20260924 13:50:28.974885 29432 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5d045e5a8ede44cba24125f78123692f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d045e5a8ede44cba24125f78123692f" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 38197 } }
I20260924 13:50:28.975232 29433 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5d045e5a8ede44cba24125f78123692f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5d045e5a8ede44cba24125f78123692f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d045e5a8ede44cba24125f78123692f" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 38197 } } }
I20260924 13:50:28.975236 29434 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5d045e5a8ede44cba24125f78123692f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5d045e5a8ede44cba24125f78123692f. Latest consensus state: current_term: 1 leader_uuid: "5d045e5a8ede44cba24125f78123692f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d045e5a8ede44cba24125f78123692f" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 38197 } } }
I20260924 13:50:28.975332 29433 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5d045e5a8ede44cba24125f78123692f [sys.catalog]: This master's current role is: LEADER
I20260924 13:50:28.975356 29434 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5d045e5a8ede44cba24125f78123692f [sys.catalog]: This master's current role is: LEADER
I20260924 13:50:28.975514 29438 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 13:50:28.975996 29438 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 13:50:28.976389 13911 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJN9WRF/build/release/bin/kudu as pid 29361
I20260924 13:50:28.976485 13911 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790257629041645-13911-0/minicluster-data/master-0/wal/instance
I20260924 13:50:28.976670 13911 external_mini_cluster.cc:1428] Setting key e1acb47a2772ea34cc8513290498abac
I20260924 13:50:28.977391 29438 catalog_manager.cc:1347] Generated new cluster ID: 6d304b4b2c5c4dfc889135d5e32001ae
I20260924 13:50:28.977438 29438 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 13:50:28.981887 29438 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 13:50:28.982949 29438 catalog_manager.cc:1504] Loading token signing keys...
Sep 24 13:50:28 dist-test-slave-sjl2 krb5kdc[29348](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257828, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.149.254@KRBTEST.COM
I20260924 13:50:28.986918 29452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:50:28.977106 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56014 (local address 127.13.149.254:38197)
0924 13:50:28.977272 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0924 13:50:28.977281 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0924 13:50:28.977498 (+ 217us) server_negotiation.cc:407] Connection header received
0924 13:50:28.977633 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:50:28.977641 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:50:28.977695 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:50:28.977827 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:50:28.978520 (+ 693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:50:28.979890 (+ 1370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:50:28.981032 (+ 1142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:50:28.981673 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:50:28.981774 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:50:28.983968 (+ 2194us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:50:28.983973 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:50:28.983973 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 13:50:28.983986 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:50:28.985573 (+ 1587us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:50:28.985883 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:50:28.985886 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:50:28.985886 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:50:28.985971 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:50:28.986142 (+ 171us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:50:28.986143 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:50:28.986143 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:50:28.986353 (+ 210us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:50:28.986494 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0924 13:50:28.986568 (+ 74us) server_negotiation.cc:299] Negotiation successful
0924 13:50:28.986713 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":119,"thread_start_us":77,"threads_started":1}
I20260924 13:50:28.989564 29438 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5d045e5a8ede44cba24125f78123692f: Generated new TSK 0
I20260924 13:50:28.989722 29438 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.13.149.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.149.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.149.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.193.keytab.
Entry for principal kudu/127.13.149.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.193.keytab.
I20260924 13:50:29.027643 13911 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJN9WRF/build/release/bin/kudu
/tmp/dist-test-taskJN9WRF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790257629041645-13911-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790257629041645-13911-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790257629041645-13911-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790257629041645-13911-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.149.193:0
--local_ip_for_outbound_sockets=127.13.149.193
--webserver_interface=127.13.149.193
--webserver_port=0
--tserver_master_addrs=127.13.149.254:38197
--keytab_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.193.keytab
--principal=kudu/127.13.149.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.149.212:44125
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790257629041645-13911-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790257629041645-13911-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790257629041645-13911-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 13:50:29.123831 29455 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 13:50:29.124049 29455 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 13:50:29.124082 29455 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 13:50:29.125293 29455 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 13:50:29.125332 29455 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 13:50:29.125358 29455 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.149.193
I20260924 13:50:29.126791 29455 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.149.212:44125
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790257629041645-13911-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790257629041645-13911-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.193.keytab
--principal=kudu/127.13.149.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.149.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790257629041645-13911-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.149.193
--webserver_port=0
--tserver_master_addrs=127.13.149.254:38197
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29455
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.149.193
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790257629041645-13911-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 12:29:37 UTC on ccycloud.quasar-iqzbxn.root.comops.site
build id 4670106
I20260924 13:50:29.126984 29455 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 13:50:29.127178 29455 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 13:50:29.132169 29461 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 13:50:29.132189 29455 server_base.cc:1034] running on GCE node
W20260924 13:50:29.132246 29463 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 13:50:29.132335 29460 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 13:50:29.132526 29455 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 13:50:29.132748 29455 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 13:50:29.133880 29455 hybrid_clock.cc:648] HybridClock initialized: now 1790257829133867 us; error 30 us; skew 500 ppm
Sep 24 13:50:29 dist-test-slave-sjl2 krb5kdc[29348](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257829, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.149.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 13:50:29.136301 29455 init.cc:377] Logged in from keytab as kudu/127.13.149.193@KRBTEST.COM (short username kudu)
I20260924 13:50:29.136698 29455 webserver.cc:460] Webserver started at http://127.13.149.193:46623/ using document root <none> and password file <none>
I20260924 13:50:29.136901 29455 fs_manager.cc:359] Metadata directory not provided
I20260924 13:50:29.136976 29455 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 13:50:29.137104 29455 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 13:50:29.138134 29455 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790257629041645-13911-0/minicluster-data/ts-0/data/instance:
uuid: "efba9ffdd4eb42929192092d77763fcc"
format_stamp: "Formatted at 2026-09-24 13:50:29 on dist-test-slave-sjl2"
server_key: "eb4a105b307721dfe0e8a662ed190d66"
server_key_iv: "418a798cc85a0cd4db844acb8c5ff085"
server_key_version: "encryptionkey@0"
I20260924 13:50:29.138473 29455 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790257629041645-13911-0/minicluster-data/ts-0/wal/instance:
uuid: "efba9ffdd4eb42929192092d77763fcc"
format_stamp: "Formatted at 2026-09-24 13:50:29 on dist-test-slave-sjl2"
server_key: "eb4a105b307721dfe0e8a662ed190d66"
server_key_iv: "418a798cc85a0cd4db844acb8c5ff085"
server_key_version: "encryptionkey@0"
I20260924 13:50:29.139669 29455 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260924 13:50:29.140336 29470 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 13:50:29.140493 29455 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 13:50:29.140573 29455 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790257629041645-13911-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790257629041645-13911-0/minicluster-data/ts-0/wal
uuid: "efba9ffdd4eb42929192092d77763fcc"
format_stamp: "Formatted at 2026-09-24 13:50:29 on dist-test-slave-sjl2"
server_key: "eb4a105b307721dfe0e8a662ed190d66"
server_key_iv: "418a798cc85a0cd4db844acb8c5ff085"
server_key_version: "encryptionkey@0"
I20260924 13:50:29.140638 29455 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790257629041645-13911-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790257629041645-13911-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790257629041645-13911-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 13:50:29.149211 29455 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 13:50:29.150919 29455 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 13:50:29.151052 29455 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 13:50:29.151262 29455 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 13:50:29.151557 29455 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 13:50:29.151592 29455 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 13:50:29.151620 29455 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 13:50:29.151645 29455 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 13:50:29.157399 29455 rpc_server.cc:307] RPC server started. Bound to: 127.13.149.193:35539
I20260924 13:50:29.157446 29583 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.149.193:35539 every 8 connection(s)
I20260924 13:50:29.157747 29455 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790257629041645-13911-0/minicluster-data/ts-0/data/info.pb
I20260924 13:50:29.163074 13911 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJN9WRF/build/release/bin/kudu as pid 29455
I20260924 13:50:29.163178 13911 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790257629041645-13911-0/minicluster-data/ts-0/wal/instance
I20260924 13:50:29.163358 13911 external_mini_cluster.cc:1428] Setting key c1603a711a5d0bf5cac28c48c733274c
Sep 24 13:50:29 dist-test-slave-sjl2 krb5kdc[29348](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257829, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.149.193@KRBTEST.COM for kudu/127.13.149.254@KRBTEST.COM
I20260924 13:50:29.167469 29452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:50:29.158435 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.149.193:38941 (local address 127.13.149.254:38197)
0924 13:50:29.158498 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0924 13:50:29.158500 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:50:29.158706 (+ 206us) server_negotiation.cc:407] Connection header received
0924 13:50:29.159251 (+ 545us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:50:29.159252 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:50:29.159265 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:50:29.159312 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:50:29.160246 (+ 934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:50:29.160764 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:50:29.161794 (+ 1030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:50:29.161999 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:50:29.162055 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:50:29.164522 (+ 2467us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:50:29.164523 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:50:29.164524 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 13:50:29.164531 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:50:29.166079 (+ 1548us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:50:29.166495 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:50:29.166496 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:50:29.166496 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:50:29.166554 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:50:29.166852 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:50:29.166852 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:50:29.166853 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:50:29.166962 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:50:29.167013 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0924 13:50:29.167318 (+ 305us) server_negotiation.cc:299] Negotiation successful
0924 13:50:29.167400 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260924 13:50:29.167737 29586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:50:29.158419 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:38197 (local address 127.13.149.193:38941)
0924 13:50:29.158580 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0924 13:50:29.158616 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0924 13:50:29.159104 (+ 488us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:50:29.159390 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:50:29.159394 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:50:29.159943 (+ 549us) client_negotiation.cc:189] Negotiated authn=SASL
0924 13:50:29.160147 (+ 204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:50:29.160152 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:50:29.160850 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:50:29.160851 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:50:29.161706 (+ 855us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:50:29.161709 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:50:29.162071 (+ 362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:50:29.162072 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:50:29.162211 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:50:29.162738 (+ 527us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 13:50:29.162747 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0924 13:50:29.164430 (+ 1683us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 13:50:29.166169 (+ 1739us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 13:50:29.166176 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 13:50:29.166178 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 13:50:29.166412 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 13:50:29.166628 (+ 216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 13:50:29.166629 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 13:50:29.166629 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 13:50:29.166784 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 13:50:29.167016 (+ 232us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 13:50:29.167022 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 13:50:29.167192 (+ 170us) client_negotiation.cc:769] Sending connection context
0924 13:50:29.167302 (+ 110us) client_negotiation.cc:240] Negotiation successful
0924 13:50:29.167471 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":75,"threads_started":1}
I20260924 13:50:29.168236 29584 heartbeater.cc:347] Connected to a master server at 127.13.149.254:38197
I20260924 13:50:29.168335 29584 heartbeater.cc:464] Registering TS with master...
I20260924 13:50:29.168511 29584 heartbeater.cc:511] Master 127.13.149.254:38197 requested a full tablet report, sending...
I20260924 13:50:29.169148 29393 ts_manager.cc:194] Registered new tserver with Master: efba9ffdd4eb42929192092d77763fcc (127.13.149.193:35539)
I20260924 13:50:29.170051 29393 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.149.193@KRBTEST.COM'} at 127.13.149.193:38941
WARNING: no policy specified for kudu/127.13.149.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.149.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.149.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.194.keytab.
Entry for principal kudu/127.13.149.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.194.keytab.
I20260924 13:50:29.189774 13911 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJN9WRF/build/release/bin/kudu
/tmp/dist-test-taskJN9WRF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790257629041645-13911-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790257629041645-13911-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790257629041645-13911-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790257629041645-13911-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.149.194:0
--local_ip_for_outbound_sockets=127.13.149.194
--webserver_interface=127.13.149.194
--webserver_port=0
--tserver_master_addrs=127.13.149.254:38197
--keytab_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.194.keytab
--principal=kudu/127.13.149.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.149.212:44125
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790257629041645-13911-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790257629041645-13911-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790257629041645-13911-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 13:50:29.269752 29589 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 13:50:29.269974 29589 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 13:50:29.270009 29589 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 13:50:29.271241 29589 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 13:50:29.271286 29589 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 13:50:29.271323 29589 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.149.194
I20260924 13:50:29.272859 29589 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.149.212:44125
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790257629041645-13911-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790257629041645-13911-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.194.keytab
--principal=kudu/127.13.149.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.149.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790257629041645-13911-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.149.194
--webserver_port=0
--tserver_master_addrs=127.13.149.254:38197
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29589
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.149.194
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790257629041645-13911-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 12:29:37 UTC on ccycloud.quasar-iqzbxn.root.comops.site
build id 4670106
I20260924 13:50:29.273119 29589 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 13:50:29.273335 29589 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 13:50:29.278303 29597 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 13:50:29.278309 29594 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 13:50:29.278359 29595 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 13:50:29.278524 29589 server_base.cc:1034] running on GCE node
I20260924 13:50:29.278775 29589 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 13:50:29.279000 29589 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 13:50:29.280156 29589 hybrid_clock.cc:648] HybridClock initialized: now 1790257829280140 us; error 28 us; skew 500 ppm
Sep 24 13:50:29 dist-test-slave-sjl2 krb5kdc[29348](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257829, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.149.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 13:50:29.282599 29589 init.cc:377] Logged in from keytab as kudu/127.13.149.194@KRBTEST.COM (short username kudu)
I20260924 13:50:29.283048 29589 webserver.cc:460] Webserver started at http://127.13.149.194:32807/ using document root <none> and password file <none>
I20260924 13:50:29.283282 29589 fs_manager.cc:359] Metadata directory not provided
I20260924 13:50:29.283331 29589 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 13:50:29.283452 29589 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 13:50:29.284505 29589 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790257629041645-13911-0/minicluster-data/ts-1/data/instance:
uuid: "dff77ad6341d44f8a14ec7f008ba9d73"
format_stamp: "Formatted at 2026-09-24 13:50:29 on dist-test-slave-sjl2"
server_key: "f20e10ce3675c422a53ee4a58a8351a5"
server_key_iv: "46011d7719e88455d00b6bad86f4363d"
server_key_version: "encryptionkey@0"
I20260924 13:50:29.284876 29589 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790257629041645-13911-0/minicluster-data/ts-1/wal/instance:
uuid: "dff77ad6341d44f8a14ec7f008ba9d73"
format_stamp: "Formatted at 2026-09-24 13:50:29 on dist-test-slave-sjl2"
server_key: "f20e10ce3675c422a53ee4a58a8351a5"
server_key_iv: "46011d7719e88455d00b6bad86f4363d"
server_key_version: "encryptionkey@0"
I20260924 13:50:29.286115 29589 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260924 13:50:29.286777 29604 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 13:50:29.286929 29589 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260924 13:50:29.287010 29589 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790257629041645-13911-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790257629041645-13911-0/minicluster-data/ts-1/wal
uuid: "dff77ad6341d44f8a14ec7f008ba9d73"
format_stamp: "Formatted at 2026-09-24 13:50:29 on dist-test-slave-sjl2"
server_key: "f20e10ce3675c422a53ee4a58a8351a5"
server_key_iv: "46011d7719e88455d00b6bad86f4363d"
server_key_version: "encryptionkey@0"
I20260924 13:50:29.287072 29589 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790257629041645-13911-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790257629041645-13911-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790257629041645-13911-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 13:50:29.316519 29589 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 13:50:29.318708 29589 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 13:50:29.318852 29589 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 13:50:29.319074 29589 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 13:50:29.319367 29589 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 13:50:29.319401 29589 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 13:50:29.319430 29589 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 13:50:29.319454 29589 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 13:50:29.325016 29589 rpc_server.cc:307] RPC server started. Bound to: 127.13.149.194:37117
I20260924 13:50:29.325060 29717 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.149.194:37117 every 8 connection(s)
I20260924 13:50:29.325376 29589 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790257629041645-13911-0/minicluster-data/ts-1/data/info.pb
Sep 24 13:50:29 dist-test-slave-sjl2 krb5kdc[29348](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257829, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.149.194@KRBTEST.COM for kudu/127.13.149.254@KRBTEST.COM
I20260924 13:50:29.334028 29452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:50:29.326068 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.149.194:54939 (local address 127.13.149.254:38197)
0924 13:50:29.326112 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0924 13:50:29.326114 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:50:29.326303 (+ 189us) server_negotiation.cc:407] Connection header received
0924 13:50:29.326789 (+ 486us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:50:29.326790 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:50:29.326800 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:50:29.326849 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:50:29.327784 (+ 935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:50:29.328300 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:50:29.329316 (+ 1016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:50:29.329513 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:50:29.329574 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:50:29.331605 (+ 2031us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:50:29.331606 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:50:29.331607 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 13:50:29.331614 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:50:29.332891 (+ 1277us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:50:29.333269 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:50:29.333269 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:50:29.333270 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:50:29.333316 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:50:29.333522 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:50:29.333523 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:50:29.333523 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:50:29.333623 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:50:29.333679 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0924 13:50:29.333899 (+ 220us) server_negotiation.cc:299] Negotiation successful
0924 13:50:29.333962 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260924 13:50:29.334221 29720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:50:29.326031 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:38197 (local address 127.13.149.194:54939)
0924 13:50:29.326181 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0924 13:50:29.326217 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0924 13:50:29.326687 (+ 470us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:50:29.326907 (+ 220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:50:29.326910 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:50:29.327475 (+ 565us) client_negotiation.cc:189] Negotiated authn=SASL
0924 13:50:29.327674 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:50:29.327678 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:50:29.328376 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:50:29.328377 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:50:29.329241 (+ 864us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:50:29.329244 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:50:29.329582 (+ 338us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:50:29.329582 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:50:29.329688 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:50:29.330181 (+ 493us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 13:50:29.330189 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0924 13:50:29.331517 (+ 1328us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 13:50:29.332983 (+ 1466us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 13:50:29.332991 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 13:50:29.332993 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 13:50:29.333206 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 13:50:29.333360 (+ 154us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 13:50:29.333360 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 13:50:29.333361 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 13:50:29.333477 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 13:50:29.333674 (+ 197us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 13:50:29.333677 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 13:50:29.333804 (+ 127us) client_negotiation.cc:769] Sending connection context
0924 13:50:29.333882 (+ 78us) client_negotiation.cc:240] Negotiation successful
0924 13:50:29.334008 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":85,"threads_started":1}
I20260924 13:50:29.334636 13911 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJN9WRF/build/release/bin/kudu as pid 29589
I20260924 13:50:29.334739 13911 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790257629041645-13911-0/minicluster-data/ts-1/wal/instance
I20260924 13:50:29.334739 29718 heartbeater.cc:347] Connected to a master server at 127.13.149.254:38197
I20260924 13:50:29.334842 29718 heartbeater.cc:464] Registering TS with master...
I20260924 13:50:29.334921 13911 external_mini_cluster.cc:1428] Setting key d8243ae41c5fee088f14ce8fa0a97b8f
I20260924 13:50:29.335035 29718 heartbeater.cc:511] Master 127.13.149.254:38197 requested a full tablet report, sending...
I20260924 13:50:29.335413 29392 ts_manager.cc:194] Registered new tserver with Master: dff77ad6341d44f8a14ec7f008ba9d73 (127.13.149.194:37117)
I20260924 13:50:29.335812 29392 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.149.194@KRBTEST.COM'} at 127.13.149.194:54939
WARNING: no policy specified for kudu/127.13.149.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.149.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.149.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.195.keytab.
Entry for principal kudu/127.13.149.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.195.keytab.
I20260924 13:50:29.359845 13911 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJN9WRF/build/release/bin/kudu
/tmp/dist-test-taskJN9WRF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790257629041645-13911-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790257629041645-13911-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790257629041645-13911-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790257629041645-13911-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.149.195:0
--local_ip_for_outbound_sockets=127.13.149.195
--webserver_interface=127.13.149.195
--webserver_port=0
--tserver_master_addrs=127.13.149.254:38197
--keytab_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.195.keytab
--principal=kudu/127.13.149.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.149.212:44125
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790257629041645-13911-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790257629041645-13911-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790257629041645-13911-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 13:50:29.439464 29723 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 13:50:29.439677 29723 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 13:50:29.439708 29723 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 13:50:29.440881 29723 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 13:50:29.440944 29723 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 13:50:29.440984 29723 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.149.195
I20260924 13:50:29.442480 29723 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.149.212:44125
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790257629041645-13911-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790257629041645-13911-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.195.keytab
--principal=kudu/127.13.149.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.149.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790257629041645-13911-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.149.195
--webserver_port=0
--tserver_master_addrs=127.13.149.254:38197
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29723
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.149.195
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790257629041645-13911-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 12:29:37 UTC on ccycloud.quasar-iqzbxn.root.comops.site
build id 4670106
I20260924 13:50:29.442726 29723 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 13:50:29.442955 29723 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 13:50:29.448048 29728 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 13:50:29.448053 29729 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 13:50:29.448288 29723 server_base.cc:1034] running on GCE node
W20260924 13:50:29.448048 29731 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 13:50:29.448625 29723 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 13:50:29.448882 29723 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 13:50:29.450063 29723 hybrid_clock.cc:648] HybridClock initialized: now 1790257829450050 us; error 27 us; skew 500 ppm
Sep 24 13:50:29 dist-test-slave-sjl2 krb5kdc[29348](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257829, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.149.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 13:50:29.453035 29723 init.cc:377] Logged in from keytab as kudu/127.13.149.195@KRBTEST.COM (short username kudu)
I20260924 13:50:29.453495 29723 webserver.cc:460] Webserver started at http://127.13.149.195:43787/ using document root <none> and password file <none>
I20260924 13:50:29.453729 29723 fs_manager.cc:359] Metadata directory not provided
I20260924 13:50:29.453787 29723 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 13:50:29.453917 29723 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 13:50:29.455180 29723 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790257629041645-13911-0/minicluster-data/ts-2/data/instance:
uuid: "1c5172f7acb64efd980cb3d3e77d1851"
format_stamp: "Formatted at 2026-09-24 13:50:29 on dist-test-slave-sjl2"
server_key: "32f6b1efa03db5795adaa2278915a8ff"
server_key_iv: "cd51cfbb7b7a2aa88053a53cfe11ba87"
server_key_version: "encryptionkey@0"
I20260924 13:50:29.455578 29723 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790257629041645-13911-0/minicluster-data/ts-2/wal/instance:
uuid: "1c5172f7acb64efd980cb3d3e77d1851"
format_stamp: "Formatted at 2026-09-24 13:50:29 on dist-test-slave-sjl2"
server_key: "32f6b1efa03db5795adaa2278915a8ff"
server_key_iv: "cd51cfbb7b7a2aa88053a53cfe11ba87"
server_key_version: "encryptionkey@0"
I20260924 13:50:29.457237 29723 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260924 13:50:29.458192 29738 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 13:50:29.458365 29723 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260924 13:50:29.458458 29723 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790257629041645-13911-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790257629041645-13911-0/minicluster-data/ts-2/wal
uuid: "1c5172f7acb64efd980cb3d3e77d1851"
format_stamp: "Formatted at 2026-09-24 13:50:29 on dist-test-slave-sjl2"
server_key: "32f6b1efa03db5795adaa2278915a8ff"
server_key_iv: "cd51cfbb7b7a2aa88053a53cfe11ba87"
server_key_version: "encryptionkey@0"
I20260924 13:50:29.458530 29723 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790257629041645-13911-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790257629041645-13911-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790257629041645-13911-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 13:50:29.469122 29723 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 13:50:29.470867 29723 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 13:50:29.470983 29723 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 13:50:29.471191 29723 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 13:50:29.471474 29723 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 13:50:29.471508 29723 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 13:50:29.471527 29723 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 13:50:29.471550 29723 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 13:50:29.477433 29723 rpc_server.cc:307] RPC server started. Bound to: 127.13.149.195:42897
I20260924 13:50:29.477473 29851 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.149.195:42897 every 8 connection(s)
I20260924 13:50:29.477767 29723 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790257629041645-13911-0/minicluster-data/ts-2/data/info.pb
Sep 24 13:50:29 dist-test-slave-sjl2 krb5kdc[29348](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257829, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.149.195@KRBTEST.COM for kudu/127.13.149.254@KRBTEST.COM
I20260924 13:50:29.484675 13911 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJN9WRF/build/release/bin/kudu as pid 29723
I20260924 13:50:29.484786 13911 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790257629041645-13911-0/minicluster-data/ts-2/wal/instance
I20260924 13:50:29.485003 13911 external_mini_cluster.cc:1428] Setting key 18dc9bc58a179f5370f0880da33f82d5
I20260924 13:50:29.486892 29452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:50:29.478536 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.149.195:58083 (local address 127.13.149.254:38197)
0924 13:50:29.478620 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0924 13:50:29.478621 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0924 13:50:29.478728 (+ 107us) server_negotiation.cc:407] Connection header received
0924 13:50:29.479199 (+ 471us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:50:29.479200 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:50:29.479211 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:50:29.479275 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:50:29.480352 (+ 1077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:50:29.480865 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:50:29.481874 (+ 1009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:50:29.482068 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:50:29.482132 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:50:29.484207 (+ 2075us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:50:29.484209 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:50:29.484209 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 13:50:29.484216 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:50:29.485723 (+ 1507us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:50:29.486111 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:50:29.486112 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:50:29.486113 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:50:29.486194 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:50:29.486407 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:50:29.486408 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:50:29.486408 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:50:29.486508 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:50:29.486551 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0924 13:50:29.486765 (+ 214us) server_negotiation.cc:299] Negotiation successful
0924 13:50:29.486828 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260924 13:50:29.487126 29854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:50:29.478444 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:38197 (local address 127.13.149.195:58083)
0924 13:50:29.478588 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0924 13:50:29.478630 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0924 13:50:29.479105 (+ 475us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:50:29.479404 (+ 299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:50:29.479408 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:50:29.479993 (+ 585us) client_negotiation.cc:189] Negotiated authn=SASL
0924 13:50:29.480252 (+ 259us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:50:29.480258 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:50:29.480987 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:50:29.480988 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:50:29.481795 (+ 807us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:50:29.481797 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:50:29.482138 (+ 341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:50:29.482138 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:50:29.482261 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:50:29.482743 (+ 482us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 13:50:29.482751 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0924 13:50:29.484107 (+ 1356us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 13:50:29.485791 (+ 1684us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 13:50:29.485800 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 13:50:29.485802 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 13:50:29.486046 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 13:50:29.486265 (+ 219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 13:50:29.486266 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 13:50:29.486266 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 13:50:29.486363 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 13:50:29.486549 (+ 186us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 13:50:29.486552 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 13:50:29.486668 (+ 116us) client_negotiation.cc:769] Sending connection context
0924 13:50:29.486744 (+ 76us) client_negotiation.cc:240] Negotiation successful
0924 13:50:29.486892 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":63,"threads_started":1}
I20260924 13:50:29.487562 29852 heartbeater.cc:347] Connected to a master server at 127.13.149.254:38197
I20260924 13:50:29.487656 29852 heartbeater.cc:464] Registering TS with master...
I20260924 13:50:29.487834 29852 heartbeater.cc:511] Master 127.13.149.254:38197 requested a full tablet report, sending...
I20260924 13:50:29.488174 29392 ts_manager.cc:194] Registered new tserver with Master: 1c5172f7acb64efd980cb3d3e77d1851 (127.13.149.195:42897)
I20260924 13:50:29.488531 29392 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.149.195@KRBTEST.COM'} at 127.13.149.195:58083
I20260924 13:50:29.496829 13911 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260924 13:50:29.503939 29452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:50:29.498264 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56024 (local address 127.13.149.254:38197)
0924 13:50:29.498349 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0924 13:50:29.498351 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:50:29.498408 (+ 57us) server_negotiation.cc:407] Connection header received
0924 13:50:29.498456 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:50:29.498457 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:50:29.498465 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:50:29.498513 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:50:29.499047 (+ 534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:50:29.499571 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:50:29.500291 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:50:29.500470 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:50:29.500515 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:50:29.501666 (+ 1151us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:50:29.501668 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:50:29.501669 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 13:50:29.501675 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:50:29.502979 (+ 1304us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:50:29.503323 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:50:29.503324 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:50:29.503324 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:50:29.503385 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:50:29.503516 (+ 131us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:50:29.503517 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:50:29.503517 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:50:29.503661 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:50:29.503745 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0924 13:50:29.503811 (+ 66us) server_negotiation.cc:299] Negotiation successful
0924 13:50:29.503870 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260924 13:50:29.505558 29392 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56024:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 13:50:29.507092 29392 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 13:50:29.513909 29866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:50:29.509790 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.193:35539 (local address 127.0.0.1:58580)
0924 13:50:29.510099 (+ 309us) negotiation.cc:107] Waiting for socket to connect
0924 13:50:29.510108 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0924 13:50:29.510162 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:50:29.510394 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:50:29.510398 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:50:29.510401 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 13:50:29.510591 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:50:29.510593 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:50:29.511442 (+ 849us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:50:29.511443 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:50:29.512602 (+ 1159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:50:29.512604 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:50:29.513594 (+ 990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:50:29.513595 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:50:29.513681 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:50:29.513699 (+ 18us) client_negotiation.cc:769] Sending connection context
0924 13:50:29.513763 (+ 64us) client_negotiation.cc:240] Negotiation successful
0924 13:50:29.513821 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":292,"thread_start_us":69,"threads_started":1}
I20260924 13:50:29.514016 29865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:50:29.509732 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37350 (local address 127.13.149.195:42897)
0924 13:50:29.510260 (+ 528us) server_negotiation.cc:206] Beginning negotiation
0924 13:50:29.510265 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 13:50:29.510274 (+ 9us) server_negotiation.cc:407] Connection header received
0924 13:50:29.510312 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:50:29.510314 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:50:29.510330 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:50:29.510477 (+ 147us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 13:50:29.510540 (+ 63us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:50:29.511702 (+ 1162us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:50:29.512903 (+ 1201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:50:29.513666 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:50:29.513737 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:50:29.513784 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0924 13:50:29.513893 (+ 109us) server_negotiation.cc:299] Negotiation successful
0924 13:50:29.513933 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":495,"thread_start_us":133,"threads_started":1}
I20260924 13:50:29.514248 29867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:50:29.509924 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58580 (local address 127.13.149.193:35539)
0924 13:50:29.510111 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0924 13:50:29.510117 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0924 13:50:29.510157 (+ 40us) server_negotiation.cc:407] Connection header received
0924 13:50:29.510212 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:50:29.510214 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:50:29.510230 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:50:29.510294 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 13:50:29.510644 (+ 350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:50:29.511355 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:50:29.512715 (+ 1360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:50:29.513493 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:50:29.513589 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:50:29.513650 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0924 13:50:29.514118 (+ 468us) server_negotiation.cc:299] Negotiation successful
0924 13:50:29.514168 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":146,"thread_start_us":87,"threads_started":1}
I20260924 13:50:29.514362 29863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:50:29.509596 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.195:42897 (local address 127.0.0.1:37350)
0924 13:50:29.509738 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0924 13:50:29.509781 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0924 13:50:29.509847 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:50:29.510409 (+ 562us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:50:29.510410 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:50:29.510412 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 13:50:29.510478 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:50:29.510480 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:50:29.511809 (+ 1329us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:50:29.511810 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:50:29.512817 (+ 1007us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:50:29.512819 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:50:29.513748 (+ 929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:50:29.513749 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:50:29.513823 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:50:29.513827 (+ 4us) client_negotiation.cc:769] Sending connection context
0924 13:50:29.513872 (+ 45us) client_negotiation.cc:240] Negotiation successful
0924 13:50:29.513914 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"mutex_wait_us":3,"thread_start_us":48,"threads_started":1}
I20260924 13:50:29.514770 29864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:50:29.509611 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.194:37117 (local address 127.0.0.1:50270)
0924 13:50:29.509963 (+ 352us) negotiation.cc:107] Waiting for socket to connect
0924 13:50:29.509975 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0924 13:50:29.510031 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:50:29.510425 (+ 394us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:50:29.510426 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:50:29.510429 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 13:50:29.510490 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:50:29.510492 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:50:29.512882 (+ 2390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:50:29.512882 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:50:29.513621 (+ 739us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:50:29.513622 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:50:29.514568 (+ 946us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:50:29.514569 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:50:29.514630 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:50:29.514633 (+ 3us) client_negotiation.cc:769] Sending connection context
0924 13:50:29.514676 (+ 43us) client_negotiation.cc:240] Negotiation successful
0924 13:50:29.514711 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":331,"thread_start_us":81,"threads_started":1}
I20260924 13:50:29.515041 29868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:50:29.509932 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50270 (local address 127.13.149.194:37117)
0924 13:50:29.510118 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0924 13:50:29.510121 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 13:50:29.510130 (+ 9us) server_negotiation.cc:407] Connection header received
0924 13:50:29.510182 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:50:29.510184 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:50:29.510201 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:50:29.510287 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 13:50:29.510571 (+ 284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:50:29.511709 (+ 1138us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:50:29.513716 (+ 2007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:50:29.514419 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:50:29.514483 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:50:29.514534 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0924 13:50:29.514927 (+ 393us) server_negotiation.cc:299] Negotiation successful
0924 13:50:29.514969 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":157,"thread_start_us":79,"threads_started":1}
I20260924 13:50:29.514958 29518 tablet_service.cc:1467] Processing CreateTablet for tablet d722a6e62e9a49f1988888251ef23cd0 (DEFAULT_TABLE table=test-table [id=9a4995b353d6426697cc912993ffb6c7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 13:50:29.515045 29786 tablet_service.cc:1467] Processing CreateTablet for tablet d722a6e62e9a49f1988888251ef23cd0 (DEFAULT_TABLE table=test-table [id=9a4995b353d6426697cc912993ffb6c7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 13:50:29.515321 29518 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d722a6e62e9a49f1988888251ef23cd0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 13:50:29.515370 29786 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d722a6e62e9a49f1988888251ef23cd0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 13:50:29.517401 29652 tablet_service.cc:1467] Processing CreateTablet for tablet d722a6e62e9a49f1988888251ef23cd0 (DEFAULT_TABLE table=test-table [id=9a4995b353d6426697cc912993ffb6c7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 13:50:29.517738 29652 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d722a6e62e9a49f1988888251ef23cd0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 13:50:29.518225 29870 tablet_bootstrap.cc:492] T d722a6e62e9a49f1988888251ef23cd0 P 1c5172f7acb64efd980cb3d3e77d1851: Bootstrap starting.
I20260924 13:50:29.518757 29870 tablet_bootstrap.cc:654] T d722a6e62e9a49f1988888251ef23cd0 P 1c5172f7acb64efd980cb3d3e77d1851: Neither blocks nor log segments found. Creating new log.
I20260924 13:50:29.519182 29870 log.cc:824] T d722a6e62e9a49f1988888251ef23cd0 P 1c5172f7acb64efd980cb3d3e77d1851: Log is configured to *not* fsync() on all Append() calls
I20260924 13:50:29.519467 29869 tablet_bootstrap.cc:492] T d722a6e62e9a49f1988888251ef23cd0 P efba9ffdd4eb42929192092d77763fcc: Bootstrap starting.
I20260924 13:50:29.519877 29870 tablet_bootstrap.cc:492] T d722a6e62e9a49f1988888251ef23cd0 P 1c5172f7acb64efd980cb3d3e77d1851: No bootstrap required, opened a new log
I20260924 13:50:29.519951 29870 ts_tablet_manager.cc:1397] T d722a6e62e9a49f1988888251ef23cd0 P 1c5172f7acb64efd980cb3d3e77d1851: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260924 13:50:29.519965 29869 tablet_bootstrap.cc:654] T d722a6e62e9a49f1988888251ef23cd0 P efba9ffdd4eb42929192092d77763fcc: Neither blocks nor log segments found. Creating new log.
I20260924 13:50:29.520407 29869 log.cc:824] T d722a6e62e9a49f1988888251ef23cd0 P efba9ffdd4eb42929192092d77763fcc: Log is configured to *not* fsync() on all Append() calls
I20260924 13:50:29.520901 29872 tablet_bootstrap.cc:492] T d722a6e62e9a49f1988888251ef23cd0 P dff77ad6341d44f8a14ec7f008ba9d73: Bootstrap starting.
I20260924 13:50:29.521116 29869 tablet_bootstrap.cc:492] T d722a6e62e9a49f1988888251ef23cd0 P efba9ffdd4eb42929192092d77763fcc: No bootstrap required, opened a new log
I20260924 13:50:29.521178 29869 ts_tablet_manager.cc:1397] T d722a6e62e9a49f1988888251ef23cd0 P efba9ffdd4eb42929192092d77763fcc: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260924 13:50:29.521564 29872 tablet_bootstrap.cc:654] T d722a6e62e9a49f1988888251ef23cd0 P dff77ad6341d44f8a14ec7f008ba9d73: Neither blocks nor log segments found. Creating new log.
I20260924 13:50:29.521750 29870 raft_consensus.cc:348] T d722a6e62e9a49f1988888251ef23cd0 P 1c5172f7acb64efd980cb3d3e77d1851 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c5172f7acb64efd980cb3d3e77d1851" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 42897 } } peers { permanent_uuid: "efba9ffdd4eb42929192092d77763fcc" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 35539 } } peers { permanent_uuid: "dff77ad6341d44f8a14ec7f008ba9d73" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 37117 } }
I20260924 13:50:29.521889 29870 raft_consensus.cc:374] T d722a6e62e9a49f1988888251ef23cd0 P 1c5172f7acb64efd980cb3d3e77d1851 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 13:50:29.521921 29870 raft_consensus.cc:729] T d722a6e62e9a49f1988888251ef23cd0 P 1c5172f7acb64efd980cb3d3e77d1851 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1c5172f7acb64efd980cb3d3e77d1851, State: Initialized, Role: FOLLOWER
I20260924 13:50:29.522050 29870 consensus_queue.cc:260] T d722a6e62e9a49f1988888251ef23cd0 P 1c5172f7acb64efd980cb3d3e77d1851 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c5172f7acb64efd980cb3d3e77d1851" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 42897 } } peers { permanent_uuid: "efba9ffdd4eb42929192092d77763fcc" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 35539 } } peers { permanent_uuid: "dff77ad6341d44f8a14ec7f008ba9d73" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 37117 } }
I20260924 13:50:29.522084 29872 log.cc:824] T d722a6e62e9a49f1988888251ef23cd0 P dff77ad6341d44f8a14ec7f008ba9d73: Log is configured to *not* fsync() on all Append() calls
I20260924 13:50:29.522307 29870 ts_tablet_manager.cc:1428] T d722a6e62e9a49f1988888251ef23cd0 P 1c5172f7acb64efd980cb3d3e77d1851: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260924 13:50:29.522424 29852 heartbeater.cc:503] Master 127.13.149.254:38197 was elected leader, sending a full tablet report...
I20260924 13:50:29.522850 29872 tablet_bootstrap.cc:492] T d722a6e62e9a49f1988888251ef23cd0 P dff77ad6341d44f8a14ec7f008ba9d73: No bootstrap required, opened a new log
I20260924 13:50:29.522922 29872 ts_tablet_manager.cc:1397] T d722a6e62e9a49f1988888251ef23cd0 P dff77ad6341d44f8a14ec7f008ba9d73: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260924 13:50:29.523138 29869 raft_consensus.cc:348] T d722a6e62e9a49f1988888251ef23cd0 P efba9ffdd4eb42929192092d77763fcc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c5172f7acb64efd980cb3d3e77d1851" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 42897 } } peers { permanent_uuid: "efba9ffdd4eb42929192092d77763fcc" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 35539 } } peers { permanent_uuid: "dff77ad6341d44f8a14ec7f008ba9d73" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 37117 } }
I20260924 13:50:29.523301 29869 raft_consensus.cc:374] T d722a6e62e9a49f1988888251ef23cd0 P efba9ffdd4eb42929192092d77763fcc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 13:50:29.523332 29869 raft_consensus.cc:729] T d722a6e62e9a49f1988888251ef23cd0 P efba9ffdd4eb42929192092d77763fcc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efba9ffdd4eb42929192092d77763fcc, State: Initialized, Role: FOLLOWER
I20260924 13:50:29.523429 29869 consensus_queue.cc:260] T d722a6e62e9a49f1988888251ef23cd0 P efba9ffdd4eb42929192092d77763fcc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c5172f7acb64efd980cb3d3e77d1851" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 42897 } } peers { permanent_uuid: "efba9ffdd4eb42929192092d77763fcc" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 35539 } } peers { permanent_uuid: "dff77ad6341d44f8a14ec7f008ba9d73" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 37117 } }
I20260924 13:50:29.523692 29869 ts_tablet_manager.cc:1428] T d722a6e62e9a49f1988888251ef23cd0 P efba9ffdd4eb42929192092d77763fcc: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260924 13:50:29.523708 29584 heartbeater.cc:503] Master 127.13.149.254:38197 was elected leader, sending a full tablet report...
I20260924 13:50:29.524681 29872 raft_consensus.cc:348] T d722a6e62e9a49f1988888251ef23cd0 P dff77ad6341d44f8a14ec7f008ba9d73 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c5172f7acb64efd980cb3d3e77d1851" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 42897 } } peers { permanent_uuid: "efba9ffdd4eb42929192092d77763fcc" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 35539 } } peers { permanent_uuid: "dff77ad6341d44f8a14ec7f008ba9d73" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 37117 } }
I20260924 13:50:29.524806 29872 raft_consensus.cc:374] T d722a6e62e9a49f1988888251ef23cd0 P dff77ad6341d44f8a14ec7f008ba9d73 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 13:50:29.524827 29872 raft_consensus.cc:729] T d722a6e62e9a49f1988888251ef23cd0 P dff77ad6341d44f8a14ec7f008ba9d73 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dff77ad6341d44f8a14ec7f008ba9d73, State: Initialized, Role: FOLLOWER
I20260924 13:50:29.524963 29872 consensus_queue.cc:260] T d722a6e62e9a49f1988888251ef23cd0 P dff77ad6341d44f8a14ec7f008ba9d73 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c5172f7acb64efd980cb3d3e77d1851" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 42897 } } peers { permanent_uuid: "efba9ffdd4eb42929192092d77763fcc" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 35539 } } peers { permanent_uuid: "dff77ad6341d44f8a14ec7f008ba9d73" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 37117 } }
I20260924 13:50:29.525202 29872 ts_tablet_manager.cc:1428] T d722a6e62e9a49f1988888251ef23cd0 P dff77ad6341d44f8a14ec7f008ba9d73: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260924 13:50:29.525241 29718 heartbeater.cc:503] Master 127.13.149.254:38197 was elected leader, sending a full tablet report...
W20260924 13:50:29.575911 29719 tablet.cc:2369] T d722a6e62e9a49f1988888251ef23cd0 P dff77ad6341d44f8a14ec7f008ba9d73: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260924 13:50:29.658448 29585 tablet.cc:2369] T d722a6e62e9a49f1988888251ef23cd0 P efba9ffdd4eb42929192092d77763fcc: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260924 13:50:29.711285 29874 raft_consensus.cc:482] T d722a6e62e9a49f1988888251ef23cd0 P 1c5172f7acb64efd980cb3d3e77d1851 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 13:50:29.711448 29874 raft_consensus.cc:504] T d722a6e62e9a49f1988888251ef23cd0 P 1c5172f7acb64efd980cb3d3e77d1851 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c5172f7acb64efd980cb3d3e77d1851" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 42897 } } peers { permanent_uuid: "efba9ffdd4eb42929192092d77763fcc" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 35539 } } peers { permanent_uuid: "dff77ad6341d44f8a14ec7f008ba9d73" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 37117 } }
I20260924 13:50:29.711819 29874 leader_election.cc:288] T d722a6e62e9a49f1988888251ef23cd0 P 1c5172f7acb64efd980cb3d3e77d1851 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers efba9ffdd4eb42929192092d77763fcc (127.13.149.193:35539), dff77ad6341d44f8a14ec7f008ba9d73 (127.13.149.194:37117)
I20260924 13:50:29.714721 29867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:50:29.712142 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.149.195:32837 (local address 127.13.149.193:35539)
0924 13:50:29.712186 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0924 13:50:29.712188 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:50:29.712221 (+ 33us) server_negotiation.cc:407] Connection header received
0924 13:50:29.712252 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:50:29.712252 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:50:29.712261 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:50:29.712318 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 13:50:29.712477 (+ 159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:50:29.713023 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:50:29.713968 (+ 945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:50:29.714378 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:50:29.714441 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:50:29.714451 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0924 13:50:29.714606 (+ 155us) server_negotiation.cc:299] Negotiation successful
0924 13:50:29.714647 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260924 13:50:29.714728 29868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:50:29.712099 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.149.195:44123 (local address 127.13.149.194:37117)
0924 13:50:29.712149 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0924 13:50:29.712152 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 13:50:29.712329 (+ 177us) server_negotiation.cc:407] Connection header received
0924 13:50:29.712360 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:50:29.712361 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:50:29.712371 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:50:29.712430 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 13:50:29.712573 (+ 143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:50:29.713119 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:50:29.714036 (+ 917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:50:29.714426 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:50:29.714501 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:50:29.714510 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0924 13:50:29.714633 (+ 123us) server_negotiation.cc:299] Negotiation successful
0924 13:50:29.714669 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260924 13:50:29.714728 29854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:50:29.712008 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.193:35539 (local address 127.13.149.195:32837)
0924 13:50:29.712104 (+ 96us) negotiation.cc:107] Waiting for socket to connect
0924 13:50:29.712117 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0924 13:50:29.712184 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:50:29.712330 (+ 146us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:50:29.712332 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:50:29.712333 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 13:50:29.712408 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:50:29.712409 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:50:29.713094 (+ 685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:50:29.713095 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:50:29.713877 (+ 782us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:50:29.713880 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:50:29.714457 (+ 577us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:50:29.714458 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:50:29.714534 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:50:29.714538 (+ 4us) client_negotiation.cc:769] Sending connection context
0924 13:50:29.714598 (+ 60us) client_negotiation.cc:240] Negotiation successful
0924 13:50:29.714640 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":72}
I20260924 13:50:29.714744 29878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:50:29.712016 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.194:37117 (local address 127.13.149.195:44123)
0924 13:50:29.712257 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0924 13:50:29.712267 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 13:50:29.712319 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:50:29.712438 (+ 119us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:50:29.712439 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:50:29.712440 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 13:50:29.712511 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:50:29.712512 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:50:29.713203 (+ 691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:50:29.713204 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:50:29.713944 (+ 740us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:50:29.713947 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:50:29.714507 (+ 560us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:50:29.714508 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:50:29.714561 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:50:29.714565 (+ 4us) client_negotiation.cc:769] Sending connection context
0924 13:50:29.714617 (+ 52us) client_negotiation.cc:240] Negotiation successful
0924 13:50:29.714652 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":225,"mutex_wait_us":68,"thread_start_us":123,"threads_started":1}
I20260924 13:50:29.715056 29538 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d722a6e62e9a49f1988888251ef23cd0" candidate_uuid: "1c5172f7acb64efd980cb3d3e77d1851" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efba9ffdd4eb42929192092d77763fcc" is_pre_election: true
I20260924 13:50:29.715070 29672 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d722a6e62e9a49f1988888251ef23cd0" candidate_uuid: "1c5172f7acb64efd980cb3d3e77d1851" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dff77ad6341d44f8a14ec7f008ba9d73" is_pre_election: true
I20260924 13:50:29.715219 29538 raft_consensus.cc:2454] T d722a6e62e9a49f1988888251ef23cd0 P efba9ffdd4eb42929192092d77763fcc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1c5172f7acb64efd980cb3d3e77d1851 in term 0.
I20260924 13:50:29.715219 29672 raft_consensus.cc:2454] T d722a6e62e9a49f1988888251ef23cd0 P dff77ad6341d44f8a14ec7f008ba9d73 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1c5172f7acb64efd980cb3d3e77d1851 in term 0.
I20260924 13:50:29.715437 29741 leader_election.cc:302] T d722a6e62e9a49f1988888251ef23cd0 P 1c5172f7acb64efd980cb3d3e77d1851 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 1c5172f7acb64efd980cb3d3e77d1851, dff77ad6341d44f8a14ec7f008ba9d73, efba9ffdd4eb42929192092d77763fcc; no voters:
I20260924 13:50:29.715564 29874 raft_consensus.cc:2781] T d722a6e62e9a49f1988888251ef23cd0 P 1c5172f7acb64efd980cb3d3e77d1851 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 13:50:29.715624 29874 raft_consensus.cc:482] T d722a6e62e9a49f1988888251ef23cd0 P 1c5172f7acb64efd980cb3d3e77d1851 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 13:50:29.715642 29874 raft_consensus.cc:3037] T d722a6e62e9a49f1988888251ef23cd0 P 1c5172f7acb64efd980cb3d3e77d1851 [term 0 FOLLOWER]: Advancing to term 1
I20260924 13:50:29.716306 29874 raft_consensus.cc:504] T d722a6e62e9a49f1988888251ef23cd0 P 1c5172f7acb64efd980cb3d3e77d1851 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c5172f7acb64efd980cb3d3e77d1851" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 42897 } } peers { permanent_uuid: "efba9ffdd4eb42929192092d77763fcc" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 35539 } } peers { permanent_uuid: "dff77ad6341d44f8a14ec7f008ba9d73" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 37117 } }
I20260924 13:50:29.716418 29874 leader_election.cc:288] T d722a6e62e9a49f1988888251ef23cd0 P 1c5172f7acb64efd980cb3d3e77d1851 [CANDIDATE]: Term 1 election: Requested vote from peers efba9ffdd4eb42929192092d77763fcc (127.13.149.193:35539), dff77ad6341d44f8a14ec7f008ba9d73 (127.13.149.194:37117)
I20260924 13:50:29.716578 29538 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d722a6e62e9a49f1988888251ef23cd0" candidate_uuid: "1c5172f7acb64efd980cb3d3e77d1851" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efba9ffdd4eb42929192092d77763fcc"
I20260924 13:50:29.716645 29538 raft_consensus.cc:3037] T d722a6e62e9a49f1988888251ef23cd0 P efba9ffdd4eb42929192092d77763fcc [term 0 FOLLOWER]: Advancing to term 1
I20260924 13:50:29.716625 29672 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d722a6e62e9a49f1988888251ef23cd0" candidate_uuid: "1c5172f7acb64efd980cb3d3e77d1851" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dff77ad6341d44f8a14ec7f008ba9d73"
I20260924 13:50:29.716711 29672 raft_consensus.cc:3037] T d722a6e62e9a49f1988888251ef23cd0 P dff77ad6341d44f8a14ec7f008ba9d73 [term 0 FOLLOWER]: Advancing to term 1
I20260924 13:50:29.717466 29538 raft_consensus.cc:2454] T d722a6e62e9a49f1988888251ef23cd0 P efba9ffdd4eb42929192092d77763fcc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1c5172f7acb64efd980cb3d3e77d1851 in term 1.
I20260924 13:50:29.717494 29672 raft_consensus.cc:2454] T d722a6e62e9a49f1988888251ef23cd0 P dff77ad6341d44f8a14ec7f008ba9d73 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1c5172f7acb64efd980cb3d3e77d1851 in term 1.
I20260924 13:50:29.717630 29741 leader_election.cc:302] T d722a6e62e9a49f1988888251ef23cd0 P 1c5172f7acb64efd980cb3d3e77d1851 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1c5172f7acb64efd980cb3d3e77d1851, dff77ad6341d44f8a14ec7f008ba9d73; no voters:
I20260924 13:50:29.717707 29874 raft_consensus.cc:2781] T d722a6e62e9a49f1988888251ef23cd0 P 1c5172f7acb64efd980cb3d3e77d1851 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 13:50:29.717837 29874 raft_consensus.cc:686] T d722a6e62e9a49f1988888251ef23cd0 P 1c5172f7acb64efd980cb3d3e77d1851 [term 1 LEADER]: Becoming Leader. State: Replica: 1c5172f7acb64efd980cb3d3e77d1851, State: Running, Role: LEADER
I20260924 13:50:29.717921 29874 consensus_queue.cc:237] T d722a6e62e9a49f1988888251ef23cd0 P 1c5172f7acb64efd980cb3d3e77d1851 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c5172f7acb64efd980cb3d3e77d1851" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 42897 } } peers { permanent_uuid: "efba9ffdd4eb42929192092d77763fcc" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 35539 } } peers { permanent_uuid: "dff77ad6341d44f8a14ec7f008ba9d73" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 37117 } }
I20260924 13:50:29.718782 29393 catalog_manager.cc:5495] T d722a6e62e9a49f1988888251ef23cd0 P 1c5172f7acb64efd980cb3d3e77d1851 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1c5172f7acb64efd980cb3d3e77d1851 (127.13.149.195). New cstate: current_term: 1 leader_uuid: "1c5172f7acb64efd980cb3d3e77d1851" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c5172f7acb64efd980cb3d3e77d1851" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 42897 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efba9ffdd4eb42929192092d77763fcc" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 35539 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dff77ad6341d44f8a14ec7f008ba9d73" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 37117 } health_report { overall_health: UNKNOWN } } }
W20260924 13:50:29.728334 29853 tablet.cc:2369] T d722a6e62e9a49f1988888251ef23cd0 P 1c5172f7acb64efd980cb3d3e77d1851: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260924 13:50:29.739437 29865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:50:29.737259 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37354 (local address 127.13.149.195:42897)
0924 13:50:29.737302 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0924 13:50:29.737304 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:50:29.737324 (+ 20us) server_negotiation.cc:407] Connection header received
0924 13:50:29.737382 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:50:29.737383 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:50:29.737402 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:50:29.737467 (+ 65us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 13:50:29.737587 (+ 120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:50:29.738044 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:50:29.738731 (+ 687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:50:29.738914 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:50:29.738966 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:50:29.739066 (+ 100us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 13:50:29.739187 (+ 121us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 13:50:29.739246 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0924 13:50:29.739332 (+ 86us) server_negotiation.cc:299] Negotiation successful
0924 13:50:29.739349 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260924 13:50:29.741231 29672 raft_consensus.cc:1261] T d722a6e62e9a49f1988888251ef23cd0 P dff77ad6341d44f8a14ec7f008ba9d73 [term 1 FOLLOWER]: Refusing update from remote peer 1c5172f7acb64efd980cb3d3e77d1851: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 13:50:29.741255 29538 raft_consensus.cc:1261] T d722a6e62e9a49f1988888251ef23cd0 P efba9ffdd4eb42929192092d77763fcc [term 1 FOLLOWER]: Refusing update from remote peer 1c5172f7acb64efd980cb3d3e77d1851: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 13:50:29.741583 29874 consensus_queue.cc:1035] T d722a6e62e9a49f1988888251ef23cd0 P 1c5172f7acb64efd980cb3d3e77d1851 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efba9ffdd4eb42929192092d77763fcc" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 35539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 13:50:29.741731 29879 consensus_queue.cc:1035] T d722a6e62e9a49f1988888251ef23cd0 P 1c5172f7acb64efd980cb3d3e77d1851 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dff77ad6341d44f8a14ec7f008ba9d73" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 37117 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 13:50:29.744292 29882 mvcc.cc:205] Tried to move back new op lower bound from 7332896070617169920 to 7332896070525239296. Current Snapshot: MvccSnapshot[applied={T|T < 7332896070617169920 or (T in {7332896070617169920})}]
I20260924 13:50:29.744407 29883 mvcc.cc:205] Tried to move back new op lower bound from 7332896070617169920 to 7332896070525239296. Current Snapshot: MvccSnapshot[applied={T|T < 7332896070617169920}]
I20260924 13:50:29.744765 29884 mvcc.cc:205] Tried to move back new op lower bound from 7332896070617169920 to 7332896070525239296. Current Snapshot: MvccSnapshot[applied={T|T < 7332896070617169920}]
I20260924 13:50:29.746119 29392 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56024:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 13:50:29.746209 29392 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56024:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 13:50:29.746935 29392 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5d045e5a8ede44cba24125f78123692f: Sending DeleteTablet for 3 replicas of tablet d722a6e62e9a49f1988888251ef23cd0
I20260924 13:50:29.747212 29786 tablet_service.cc:1514] Processing DeleteTablet for tablet d722a6e62e9a49f1988888251ef23cd0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 13:50:29 UTC) from {username='kudu', principal='kudu/127.13.149.254@KRBTEST.COM'} at 127.0.0.1:37350
I20260924 13:50:29.747256 29518 tablet_service.cc:1514] Processing DeleteTablet for tablet d722a6e62e9a49f1988888251ef23cd0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 13:50:29 UTC) from {username='kudu', principal='kudu/127.13.149.254@KRBTEST.COM'} at 127.0.0.1:58580
I20260924 13:50:29.747313 13911 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJN9WRF/build/release/bin/kudu with pid 29455
I20260924 13:50:29.747430 29892 tablet_replica.cc:330] T d722a6e62e9a49f1988888251ef23cd0 P 1c5172f7acb64efd980cb3d3e77d1851: stopping tablet replica
I20260924 13:50:29.747511 29892 raft_consensus.cc:2229] T d722a6e62e9a49f1988888251ef23cd0 P 1c5172f7acb64efd980cb3d3e77d1851 [term 1 LEADER]: Raft consensus shutting down.
I20260924 13:50:29.747582 29892 raft_consensus.cc:2258] T d722a6e62e9a49f1988888251ef23cd0 P 1c5172f7acb64efd980cb3d3e77d1851 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 13:50:29.747812 29652 tablet_service.cc:1514] Processing DeleteTablet for tablet d722a6e62e9a49f1988888251ef23cd0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 13:50:29 UTC) from {username='kudu', principal='kudu/127.13.149.254@KRBTEST.COM'} at 127.0.0.1:50270
I20260924 13:50:29.747979 29894 tablet_replica.cc:330] T d722a6e62e9a49f1988888251ef23cd0 P dff77ad6341d44f8a14ec7f008ba9d73: stopping tablet replica
I20260924 13:50:29.748045 29894 raft_consensus.cc:2229] T d722a6e62e9a49f1988888251ef23cd0 P dff77ad6341d44f8a14ec7f008ba9d73 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 13:50:29.748111 29894 raft_consensus.cc:2258] T d722a6e62e9a49f1988888251ef23cd0 P dff77ad6341d44f8a14ec7f008ba9d73 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 13:50:29.748461 29894 ts_tablet_manager.cc:1905] T d722a6e62e9a49f1988888251ef23cd0 P dff77ad6341d44f8a14ec7f008ba9d73: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 13:50:29.749744 29892 ts_tablet_manager.cc:1905] T d722a6e62e9a49f1988888251ef23cd0 P 1c5172f7acb64efd980cb3d3e77d1851: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 13:50:29.750025 29894 ts_tablet_manager.cc:1918] T d722a6e62e9a49f1988888251ef23cd0 P dff77ad6341d44f8a14ec7f008ba9d73: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 13:50:29.750083 29894 log.cc:1196] T d722a6e62e9a49f1988888251ef23cd0 P dff77ad6341d44f8a14ec7f008ba9d73: Deleting WAL directory at /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790257629041645-13911-0/minicluster-data/ts-1/wal/wals/d722a6e62e9a49f1988888251ef23cd0
I20260924 13:50:29.750334 29894 ts_tablet_manager.cc:1939] T d722a6e62e9a49f1988888251ef23cd0 P dff77ad6341d44f8a14ec7f008ba9d73: Deleting consensus metadata
I20260924 13:50:29.750768 29379 catalog_manager.cc:4841] TS dff77ad6341d44f8a14ec7f008ba9d73 (127.13.149.194:37117): tablet d722a6e62e9a49f1988888251ef23cd0 (table test-table [id=9a4995b353d6426697cc912993ffb6c7]) successfully deleted
I20260924 13:50:29.751112 29892 ts_tablet_manager.cc:1918] T d722a6e62e9a49f1988888251ef23cd0 P 1c5172f7acb64efd980cb3d3e77d1851: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 13:50:29.751157 29892 log.cc:1196] T d722a6e62e9a49f1988888251ef23cd0 P 1c5172f7acb64efd980cb3d3e77d1851: Deleting WAL directory at /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790257629041645-13911-0/minicluster-data/ts-2/wal/wals/d722a6e62e9a49f1988888251ef23cd0
I20260924 13:50:29.751379 29892 ts_tablet_manager.cc:1939] T d722a6e62e9a49f1988888251ef23cd0 P 1c5172f7acb64efd980cb3d3e77d1851: Deleting consensus metadata
I20260924 13:50:29.751725 29377 catalog_manager.cc:4841] TS 1c5172f7acb64efd980cb3d3e77d1851 (127.13.149.195:42897): tablet d722a6e62e9a49f1988888251ef23cd0 (table test-table [id=9a4995b353d6426697cc912993ffb6c7]) successfully deleted
W20260924 13:50:29.754556 29377 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.13.149.193:35539 (error 108)
I20260924 13:50:29.754815 13911 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJN9WRF/build/release/bin/kudu with pid 29589
I20260924 13:50:29.760146 13911 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJN9WRF/build/release/bin/kudu with pid 29723
I20260924 13:50:29.760300 29863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:50:29.760147 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.193:35539 (local address 127.0.0.1:58582)
0924 13:50:29.760218 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0924 13:50:29.760265 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.149.193:35539: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260924 13:50:29.761238 29377 catalog_manager.cc:4584] TS efba9ffdd4eb42929192092d77763fcc (127.13.149.193:35539): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet d722a6e62e9a49f1988888251ef23cd0: Network error: Client connection negotiation failed: client connection to 127.13.149.193:35539: connect: Connection refused (error 111)
I20260924 13:50:29.764256 13911 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJN9WRF/build/release/bin/kudu with pid 29361
I20260924 13:50:49.768766 13911 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJN9WRF/build/release/bin/kudu
/tmp/dist-test-taskJN9WRF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790257629041645-13911-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790257629041645-13911-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790257629041645-13911-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790257629041645-13911-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.149.254:38197
--webserver_interface=127.13.149.254
--webserver_port=34079
--keytab_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.254.keytab
--principal=kudu/127.13.149.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.149.212:44125
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.149.254:38197
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790257629041645-13911-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790257629041645-13911-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790257629041645-13911-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 13:50:49.848037 29896 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 13:50:49.848238 29896 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 13:50:49.848263 29896 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 13:50:49.849473 29896 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260924 13:50:49.849525 29896 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260924 13:50:49.849537 29896 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260924 13:50:49.849547 29896 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 13:50:49.849560 29896 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 13:50:49.849571 29896 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 13:50:49.849584 29896 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 13:50:49.849622 29896 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 13:50:49.851150 29896 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.149.212:44125
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790257629041645-13911-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790257629041645-13911-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.13.149.254:38197
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.254.keytab
--principal=kudu/127.13.149.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.149.254:38197
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790257629041645-13911-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.149.254
--webserver_port=34079
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29896
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790257629041645-13911-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 12:29:37 UTC on ccycloud.quasar-iqzbxn.root.comops.site
build id 4670106
I20260924 13:50:49.851387 29896 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 13:50:49.851614 29896 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 13:50:49.860328 29903 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 13:50:49.860328 29905 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 13:50:49.860328 29902 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 13:50:49.860569 29896 server_base.cc:1034] running on GCE node
I20260924 13:50:49.860837 29896 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 13:50:49.861177 29896 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 13:50:49.862335 29896 hybrid_clock.cc:648] HybridClock initialized: now 1790257849862319 us; error 37 us; skew 500 ppm
Sep 24 13:50:49 dist-test-slave-sjl2 krb5kdc[29348](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257849, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.149.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 13:50:49.870699 29896 init.cc:377] Logged in from keytab as kudu/127.13.149.254@KRBTEST.COM (short username kudu)
I20260924 13:50:49.871214 29896 webserver.cc:460] Webserver started at http://127.13.149.254:34079/ using document root <none> and password file <none>
I20260924 13:50:49.871460 29896 fs_manager.cc:359] Metadata directory not provided
I20260924 13:50:49.871520 29896 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 13:50:49.873013 29896 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260924 13:50:49.873713 29912 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 13:50:49.873879 29896 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260924 13:50:49.873965 29896 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790257629041645-13911-0/minicluster-data/master-0/data,/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790257629041645-13911-0/minicluster-data/master-0/wal
uuid: "5d045e5a8ede44cba24125f78123692f"
format_stamp: "Formatted at 2026-09-24 13:50:28 on dist-test-slave-sjl2"
server_key: "cb869e500d58c01ee6af39032eb28186"
server_key_iv: "3152d23837859a2898ff482ee43ee262"
server_key_version: "encryptionkey@0"
I20260924 13:50:49.874367 29896 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790257629041645-13911-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790257629041645-13911-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790257629041645-13911-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 13:50:49.895170 29896 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 13:50:49.897195 29896 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 13:50:49.897336 29896 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 13:50:49.901185 29896 rpc_server.cc:307] RPC server started. Bound to: 127.13.149.254:38197
I20260924 13:50:49.901223 29964 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.149.254:38197 every 8 connection(s)
I20260924 13:50:49.901583 29896 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790257629041645-13911-0/minicluster-data/master-0/data/info.pb
I20260924 13:50:49.903017 29965 sys_catalog.cc:263] Verifying existing consensus state
I20260924 13:50:49.903702 29965 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5d045e5a8ede44cba24125f78123692f: Bootstrap starting.
I20260924 13:50:49.904023 13911 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJN9WRF/build/release/bin/kudu as pid 29896
I20260924 13:50:49.904280 13911 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJN9WRF/build/release/bin/kudu
/tmp/dist-test-taskJN9WRF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790257629041645-13911-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790257629041645-13911-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790257629041645-13911-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790257629041645-13911-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.149.193:35539
--local_ip_for_outbound_sockets=127.13.149.193
--tserver_master_addrs=127.13.149.254:38197
--webserver_port=46623
--webserver_interface=127.13.149.193
--keytab_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.193.keytab
--principal=kudu/127.13.149.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.149.212:44125
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790257629041645-13911-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790257629041645-13911-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790257629041645-13911-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260924 13:50:49.911944 29965 log.cc:824] T 00000000000000000000000000000000 P 5d045e5a8ede44cba24125f78123692f: Log is configured to *not* fsync() on all Append() calls
I20260924 13:50:49.913965 29965 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5d045e5a8ede44cba24125f78123692f: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260924 13:50:49.914237 29965 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5d045e5a8ede44cba24125f78123692f: Bootstrap complete.
I20260924 13:50:49.916375 29965 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5d045e5a8ede44cba24125f78123692f [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d045e5a8ede44cba24125f78123692f" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 38197 } }
I20260924 13:50:49.916605 29965 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5d045e5a8ede44cba24125f78123692f [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5d045e5a8ede44cba24125f78123692f, State: Initialized, Role: FOLLOWER
I20260924 13:50:49.916756 29965 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5d045e5a8ede44cba24125f78123692f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d045e5a8ede44cba24125f78123692f" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 38197 } }
I20260924 13:50:49.916816 29965 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5d045e5a8ede44cba24125f78123692f [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 13:50:49.916862 29965 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5d045e5a8ede44cba24125f78123692f [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 13:50:49.916908 29965 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5d045e5a8ede44cba24125f78123692f [term 1 FOLLOWER]: Advancing to term 2
I20260924 13:50:49.918059 29965 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5d045e5a8ede44cba24125f78123692f [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d045e5a8ede44cba24125f78123692f" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 38197 } }
I20260924 13:50:49.918148 29965 leader_election.cc:302] T 00000000000000000000000000000000 P 5d045e5a8ede44cba24125f78123692f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5d045e5a8ede44cba24125f78123692f; no voters:
I20260924 13:50:49.918305 29965 leader_election.cc:288] T 00000000000000000000000000000000 P 5d045e5a8ede44cba24125f78123692f [CANDIDATE]: Term 2 election: Requested vote from peers
I20260924 13:50:49.918354 29969 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5d045e5a8ede44cba24125f78123692f [term 2 FOLLOWER]: Leader election won for term 2
I20260924 13:50:49.918469 29969 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5d045e5a8ede44cba24125f78123692f [term 2 LEADER]: Becoming Leader. State: Replica: 5d045e5a8ede44cba24125f78123692f, State: Running, Role: LEADER
I20260924 13:50:49.918577 29965 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5d045e5a8ede44cba24125f78123692f [sys.catalog]: configured and running, proceeding with master startup.
I20260924 13:50:49.918560 29969 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5d045e5a8ede44cba24125f78123692f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d045e5a8ede44cba24125f78123692f" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 38197 } }
I20260924 13:50:49.918916 29971 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5d045e5a8ede44cba24125f78123692f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5d045e5a8ede44cba24125f78123692f. Latest consensus state: current_term: 2 leader_uuid: "5d045e5a8ede44cba24125f78123692f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d045e5a8ede44cba24125f78123692f" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 38197 } } }
I20260924 13:50:49.919066 29971 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5d045e5a8ede44cba24125f78123692f [sys.catalog]: This master's current role is: LEADER
I20260924 13:50:49.919239 29970 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5d045e5a8ede44cba24125f78123692f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "5d045e5a8ede44cba24125f78123692f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d045e5a8ede44cba24125f78123692f" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 38197 } } }
I20260924 13:50:49.919339 29970 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5d045e5a8ede44cba24125f78123692f [sys.catalog]: This master's current role is: LEADER
I20260924 13:50:49.919447 29978 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 13:50:49.920212 29978 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 13:50:49.920320 29978 catalog_manager.cc:1259] Loaded cluster ID: 6d304b4b2c5c4dfc889135d5e32001ae
I20260924 13:50:49.920352 29978 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 13:50:49.921756 29978 catalog_manager.cc:1504] Loading token signing keys...
I20260924 13:50:49.921949 29978 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 5d045e5a8ede44cba24125f78123692f: Loaded TSK: 0
I20260924 13:50:49.952065 29978 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5d045e5a8ede44cba24125f78123692f: Generated new TSK 1
I20260924 13:50:49.952196 29978 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260924 13:50:49.990192 29967 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 13:50:49.990423 29967 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 13:50:49.990458 29967 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 13:50:49.991680 29967 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 13:50:49.991734 29967 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 13:50:49.991773 29967 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.149.193
I20260924 13:50:49.993495 29967 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.149.212:44125
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790257629041645-13911-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790257629041645-13911-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.193.keytab
--principal=kudu/127.13.149.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.149.193:35539
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790257629041645-13911-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.149.193
--webserver_port=46623
--tserver_master_addrs=127.13.149.254:38197
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29967
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.149.193
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790257629041645-13911-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 12:29:37 UTC on ccycloud.quasar-iqzbxn.root.comops.site
build id 4670106
I20260924 13:50:49.993764 29967 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 13:50:49.993989 29967 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 13:50:49.999560 29995 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 13:50:49.999583 29992 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 13:50:49.999560 29993 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 13:50:49.999996 29967 server_base.cc:1034] running on GCE node
I20260924 13:50:50.000195 29967 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 13:50:50.000442 29967 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 13:50:50.001632 29967 hybrid_clock.cc:648] HybridClock initialized: now 1790257850001613 us; error 35 us; skew 500 ppm
Sep 24 13:50:49 dist-test-slave-sjl2 krb5kdc[29348](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257849, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.149.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 13:50:50.004171 29967 init.cc:377] Logged in from keytab as kudu/127.13.149.193@KRBTEST.COM (short username kudu)
I20260924 13:50:50.004875 29967 webserver.cc:460] Webserver started at http://127.13.149.193:46623/ using document root <none> and password file <none>
I20260924 13:50:50.005223 29967 fs_manager.cc:359] Metadata directory not provided
I20260924 13:50:50.005292 29967 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 13:50:50.006714 29967 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260924 13:50:50.007362 30002 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 13:50:50.007543 29967 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260924 13:50:50.007620 29967 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790257629041645-13911-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790257629041645-13911-0/minicluster-data/ts-0/wal
uuid: "efba9ffdd4eb42929192092d77763fcc"
format_stamp: "Formatted at 2026-09-24 13:50:29 on dist-test-slave-sjl2"
server_key: "eb4a105b307721dfe0e8a662ed190d66"
server_key_iv: "418a798cc85a0cd4db844acb8c5ff085"
server_key_version: "encryptionkey@0"
I20260924 13:50:50.007992 29967 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790257629041645-13911-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790257629041645-13911-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790257629041645-13911-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 13:50:50.020324 29967 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 13:50:50.022529 29967 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 13:50:50.022655 29967 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 13:50:50.022888 29967 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 13:50:50.023377 30009 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260924 13:50:50.024804 29967 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260924 13:50:50.024858 29967 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260924 13:50:50.024884 29967 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260924 13:50:50.025549 29967 ts_tablet_manager.cc:610] Registered 1 tablets
I20260924 13:50:50.025592 29967 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260924 13:50:50.025671 30009 tablet_bootstrap.cc:492] T d722a6e62e9a49f1988888251ef23cd0 P efba9ffdd4eb42929192092d77763fcc: Bootstrap starting.
I20260924 13:50:50.032976 29967 rpc_server.cc:307] RPC server started. Bound to: 127.13.149.193:35539
I20260924 13:50:50.033166 30116 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.149.193:35539 every 8 connection(s)
I20260924 13:50:50.033476 29967 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790257629041645-13911-0/minicluster-data/ts-0/data/info.pb
I20260924 13:50:50.036688 30009 log.cc:824] T d722a6e62e9a49f1988888251ef23cd0 P efba9ffdd4eb42929192092d77763fcc: Log is configured to *not* fsync() on all Append() calls
I20260924 13:50:50.039340 13911 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJN9WRF/build/release/bin/kudu as pid 29967
I20260924 13:50:50.039438 30009 tablet_bootstrap.cc:492] T d722a6e62e9a49f1988888251ef23cd0 P efba9ffdd4eb42929192092d77763fcc: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260924 13:50:50.039633 13911 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJN9WRF/build/release/bin/kudu
/tmp/dist-test-taskJN9WRF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790257629041645-13911-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790257629041645-13911-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790257629041645-13911-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790257629041645-13911-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.149.194:37117
--local_ip_for_outbound_sockets=127.13.149.194
--tserver_master_addrs=127.13.149.254:38197
--webserver_port=32807
--webserver_interface=127.13.149.194
--keytab_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.194.keytab
--principal=kudu/127.13.149.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.149.212:44125
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790257629041645-13911-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790257629041645-13911-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790257629041645-13911-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260924 13:50:50.039723 30009 tablet_bootstrap.cc:492] T d722a6e62e9a49f1988888251ef23cd0 P efba9ffdd4eb42929192092d77763fcc: Bootstrap complete.
I20260924 13:50:50.040750 30009 ts_tablet_manager.cc:1397] T d722a6e62e9a49f1988888251ef23cd0 P efba9ffdd4eb42929192092d77763fcc: Time spent bootstrapping tablet: real 0.015s user 0.000s sys 0.011s
I20260924 13:50:50.041975 30009 raft_consensus.cc:348] T d722a6e62e9a49f1988888251ef23cd0 P efba9ffdd4eb42929192092d77763fcc [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c5172f7acb64efd980cb3d3e77d1851" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 42897 } } peers { permanent_uuid: "efba9ffdd4eb42929192092d77763fcc" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 35539 } } peers { permanent_uuid: "dff77ad6341d44f8a14ec7f008ba9d73" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 37117 } }
I20260924 13:50:50.042220 30009 raft_consensus.cc:729] T d722a6e62e9a49f1988888251ef23cd0 P efba9ffdd4eb42929192092d77763fcc [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: efba9ffdd4eb42929192092d77763fcc, State: Initialized, Role: FOLLOWER
I20260924 13:50:50.042380 30009 consensus_queue.cc:260] T d722a6e62e9a49f1988888251ef23cd0 P efba9ffdd4eb42929192092d77763fcc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c5172f7acb64efd980cb3d3e77d1851" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 42897 } } peers { permanent_uuid: "efba9ffdd4eb42929192092d77763fcc" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 35539 } } peers { permanent_uuid: "dff77ad6341d44f8a14ec7f008ba9d73" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 37117 } }
I20260924 13:50:50.042678 30009 ts_tablet_manager.cc:1428] T d722a6e62e9a49f1988888251ef23cd0 P efba9ffdd4eb42929192092d77763fcc: Time spent starting tablet: real 0.002s user 0.004s sys 0.001s
Sep 24 13:50:50 dist-test-slave-sjl2 krb5kdc[29348](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257849, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.149.193@KRBTEST.COM for kudu/127.13.149.254@KRBTEST.COM
I20260924 13:50:50.050453 30119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:50:50.034204 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:38197 (local address 127.13.149.193:51579)
0924 13:50:50.036983 (+ 2779us) negotiation.cc:107] Waiting for socket to connect
0924 13:50:50.037031 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0924 13:50:50.037697 (+ 666us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:50:50.038257 (+ 560us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:50:50.038262 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:50:50.038618 (+ 356us) client_negotiation.cc:189] Negotiated authn=SASL
0924 13:50:50.038893 (+ 275us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:50:50.038900 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:50:50.040841 (+ 1941us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:50:50.040843 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:50:50.042223 (+ 1380us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:50:50.042227 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:50:50.043248 (+ 1021us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:50:50.043249 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:50:50.043397 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:50:50.044001 (+ 604us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 13:50:50.044011 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0924 13:50:50.045949 (+ 1938us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 13:50:50.048424 (+ 2475us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 13:50:50.048436 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 13:50:50.048439 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0924 13:50:50.048768 (+ 329us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 13:50:50.049117 (+ 349us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 13:50:50.049118 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 13:50:50.049119 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 13:50:50.049263 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 13:50:50.049768 (+ 505us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 13:50:50.049774 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 13:50:50.049935 (+ 161us) client_negotiation.cc:769] Sending connection context
0924 13:50:50.050038 (+ 103us) client_negotiation.cc:240] Negotiation successful
0924 13:50:50.050220 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2730,"thread_start_us":88,"threads_started":1}
I20260924 13:50:50.050649 30120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:50:50.034343 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.149.193:51579 (local address 127.13.149.254:38197)
0924 13:50:50.034532 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0924 13:50:50.034541 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0924 13:50:50.037180 (+ 2639us) server_negotiation.cc:407] Connection header received
0924 13:50:50.037991 (+ 811us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:50:50.037998 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:50:50.038057 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:50:50.038210 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:50:50.039029 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:50:50.040668 (+ 1639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:50:50.042360 (+ 1692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:50:50.043031 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:50:50.043117 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:50:50.046102 (+ 2985us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:50:50.046108 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:50:50.046108 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 13:50:50.046125 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:50:50.048311 (+ 2186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:50:50.048874 (+ 563us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:50:50.048879 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:50:50.048879 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:50:50.049050 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:50:50.049339 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:50:50.049340 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:50:50.049340 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:50:50.049694 (+ 354us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:50:50.049786 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0924 13:50:50.050138 (+ 352us) server_negotiation.cc:299] Negotiation successful
0924 13:50:50.050288 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":134,"thread_start_us":96,"threads_started":1}
I20260924 13:50:50.051616 30117 heartbeater.cc:347] Connected to a master server at 127.13.149.254:38197
I20260924 13:50:50.051736 30117 heartbeater.cc:464] Registering TS with master...
I20260924 13:50:50.051929 30117 heartbeater.cc:511] Master 127.13.149.254:38197 requested a full tablet report, sending...
I20260924 13:50:50.052747 29929 ts_manager.cc:194] Registered new tserver with Master: efba9ffdd4eb42929192092d77763fcc (127.13.149.193:35539)
I20260924 13:50:50.054724 29929 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.149.193@KRBTEST.COM'} at 127.13.149.193:51579
I20260924 13:50:50.056628 30117 heartbeater.cc:503] Master 127.13.149.254:38197 was elected leader, sending a full tablet report...
I20260924 13:50:50.065497 30124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:50:50.053459 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.193:35539 (local address 127.0.0.1:40462)
0924 13:50:50.061095 (+ 7636us) negotiation.cc:107] Waiting for socket to connect
0924 13:50:50.061154 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0924 13:50:50.061246 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:50:50.061579 (+ 333us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:50:50.061583 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:50:50.061586 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 13:50:50.061674 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:50:50.061676 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:50:50.063021 (+ 1345us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:50:50.063023 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:50:50.064169 (+ 1146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:50:50.064174 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:50:50.065145 (+ 971us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:50:50.065146 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:50:50.065249 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:50:50.065265 (+ 16us) client_negotiation.cc:769] Sending connection context
0924 13:50:50.065342 (+ 77us) client_negotiation.cc:240] Negotiation successful
0924 13:50:50.065404 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7588,"thread_start_us":119,"threads_started":1}
I20260924 13:50:50.065495 30125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:50:50.053768 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40462 (local address 127.13.149.193:35539)
0924 13:50:50.061399 (+ 7631us) server_negotiation.cc:206] Beginning negotiation
0924 13:50:50.061404 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 13:50:50.061414 (+ 10us) server_negotiation.cc:407] Connection header received
0924 13:50:50.061452 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:50:50.061454 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:50:50.061470 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:50:50.061543 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 13:50:50.061769 (+ 226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:50:50.062894 (+ 1125us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:50:50.064289 (+ 1395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:50:50.065042 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:50:50.065126 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:50:50.065185 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0924 13:50:50.065362 (+ 177us) server_negotiation.cc:299] Negotiation successful
0924 13:50:50.065409 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7591,"thread_start_us":75,"threads_started":1}
I20260924 13:50:50.066161 30051 tablet_service.cc:1514] Processing DeleteTablet for tablet d722a6e62e9a49f1988888251ef23cd0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 13:50:29 UTC) from {username='kudu', principal='kudu/127.13.149.254@KRBTEST.COM'} at 127.0.0.1:40462
I20260924 13:50:50.066995 30126 tablet_replica.cc:330] stopping tablet replica
I20260924 13:50:50.067186 30126 raft_consensus.cc:2229] T d722a6e62e9a49f1988888251ef23cd0 P efba9ffdd4eb42929192092d77763fcc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 13:50:50.067281 30126 raft_consensus.cc:2258] T d722a6e62e9a49f1988888251ef23cd0 P efba9ffdd4eb42929192092d77763fcc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 13:50:50.067761 30126 ts_tablet_manager.cc:1905] T d722a6e62e9a49f1988888251ef23cd0 P efba9ffdd4eb42929192092d77763fcc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 13:50:50.070067 30126 ts_tablet_manager.cc:1918] T d722a6e62e9a49f1988888251ef23cd0 P efba9ffdd4eb42929192092d77763fcc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 13:50:50.070160 30126 log.cc:1196] T d722a6e62e9a49f1988888251ef23cd0 P efba9ffdd4eb42929192092d77763fcc: Deleting WAL directory at /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790257629041645-13911-0/minicluster-data/ts-0/wal/wals/d722a6e62e9a49f1988888251ef23cd0
I20260924 13:50:50.070360 30126 ts_tablet_manager.cc:1939] T d722a6e62e9a49f1988888251ef23cd0 P efba9ffdd4eb42929192092d77763fcc: Deleting consensus metadata
I20260924 13:50:50.070883 29913 catalog_manager.cc:4841] TS efba9ffdd4eb42929192092d77763fcc (127.13.149.193:35539): tablet d722a6e62e9a49f1988888251ef23cd0 (table test-table [id=9a4995b353d6426697cc912993ffb6c7]) successfully deleted
W20260924 13:50:50.127978 30122 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 13:50:50.128201 30122 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 13:50:50.128226 30122 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 13:50:50.129650 30122 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 13:50:50.129714 30122 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 13:50:50.129756 30122 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.149.194
I20260924 13:50:50.131431 30122 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.149.212:44125
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790257629041645-13911-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790257629041645-13911-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.194.keytab
--principal=kudu/127.13.149.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.149.194:37117
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790257629041645-13911-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.149.194
--webserver_port=32807
--tserver_master_addrs=127.13.149.254:38197
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30122
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.149.194
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790257629041645-13911-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 12:29:37 UTC on ccycloud.quasar-iqzbxn.root.comops.site
build id 4670106
I20260924 13:50:50.131713 30122 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 13:50:50.131948 30122 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 13:50:50.137219 30132 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 13:50:50.137238 30134 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 13:50:50.137310 30131 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 13:50:50.137293 30122 server_base.cc:1034] running on GCE node
I20260924 13:50:50.137684 30122 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 13:50:50.137921 30122 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 13:50:50.139065 30122 hybrid_clock.cc:648] HybridClock initialized: now 1790257850139050 us; error 30 us; skew 500 ppm
Sep 24 13:50:50 dist-test-slave-sjl2 krb5kdc[29348](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257850, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.149.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 13:50:50.141513 30122 init.cc:377] Logged in from keytab as kudu/127.13.149.194@KRBTEST.COM (short username kudu)
I20260924 13:50:50.141954 30122 webserver.cc:460] Webserver started at http://127.13.149.194:32807/ using document root <none> and password file <none>
I20260924 13:50:50.142168 30122 fs_manager.cc:359] Metadata directory not provided
I20260924 13:50:50.142228 30122 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 13:50:50.143494 30122 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260924 13:50:50.143989 30141 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 13:50:50.144131 30122 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260924 13:50:50.144210 30122 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790257629041645-13911-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790257629041645-13911-0/minicluster-data/ts-1/wal
uuid: "dff77ad6341d44f8a14ec7f008ba9d73"
format_stamp: "Formatted at 2026-09-24 13:50:29 on dist-test-slave-sjl2"
server_key: "f20e10ce3675c422a53ee4a58a8351a5"
server_key_iv: "46011d7719e88455d00b6bad86f4363d"
server_key_version: "encryptionkey@0"
I20260924 13:50:50.144531 30122 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790257629041645-13911-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790257629041645-13911-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790257629041645-13911-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 13:50:50.159719 30122 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 13:50:50.161881 30122 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 13:50:50.162020 30122 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 13:50:50.162250 30122 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 13:50:50.162540 30122 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 13:50:50.162572 30122 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 13:50:50.162601 30122 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 13:50:50.162632 30122 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 13:50:50.168393 30122 rpc_server.cc:307] RPC server started. Bound to: 127.13.149.194:37117
I20260924 13:50:50.168467 30254 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.149.194:37117 every 8 connection(s)
I20260924 13:50:50.168782 30122 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790257629041645-13911-0/minicluster-data/ts-1/data/info.pb
Sep 24 13:50:50 dist-test-slave-sjl2 krb5kdc[29348](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257850, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.149.194@KRBTEST.COM for kudu/127.13.149.254@KRBTEST.COM
I20260924 13:50:50.176860 13911 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJN9WRF/build/release/bin/kudu as pid 30122
I20260924 13:50:50.177193 13911 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJN9WRF/build/release/bin/kudu
/tmp/dist-test-taskJN9WRF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790257629041645-13911-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790257629041645-13911-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790257629041645-13911-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790257629041645-13911-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.149.195:42897
--local_ip_for_outbound_sockets=127.13.149.195
--tserver_master_addrs=127.13.149.254:38197
--webserver_port=43787
--webserver_interface=127.13.149.195
--keytab_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.195.keytab
--principal=kudu/127.13.149.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.149.212:44125
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790257629041645-13911-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790257629041645-13911-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790257629041645-13911-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260924 13:50:50.177716 30120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:50:50.169569 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.149.194:56305 (local address 127.13.149.254:38197)
0924 13:50:50.169646 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0924 13:50:50.169647 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0924 13:50:50.169839 (+ 192us) server_negotiation.cc:407] Connection header received
0924 13:50:50.170361 (+ 522us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:50:50.170363 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:50:50.170377 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:50:50.170424 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:50:50.171380 (+ 956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:50:50.171884 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:50:50.172858 (+ 974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:50:50.173113 (+ 255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:50:50.173171 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:50:50.175251 (+ 2080us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:50:50.175252 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:50:50.175253 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 13:50:50.175261 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:50:50.176321 (+ 1060us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:50:50.176696 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:50:50.176697 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:50:50.176698 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:50:50.176752 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:50:50.176988 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:50:50.176989 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:50:50.176989 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:50:50.177106 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:50:50.177167 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0924 13:50:50.177480 (+ 313us) server_negotiation.cc:299] Negotiation successful
0924 13:50:50.177622 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260924 13:50:50.177753 30257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:50:50.169551 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:38197 (local address 127.13.149.194:56305)
0924 13:50:50.169723 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0924 13:50:50.169759 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0924 13:50:50.170260 (+ 501us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:50:50.170508 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:50:50.170512 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:50:50.171075 (+ 563us) client_negotiation.cc:189] Negotiated authn=SASL
0924 13:50:50.171287 (+ 212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:50:50.171291 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:50:50.171968 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:50:50.171969 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:50:50.172776 (+ 807us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:50:50.172778 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:50:50.173184 (+ 406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:50:50.173185 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:50:50.173310 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:50:50.173794 (+ 484us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 13:50:50.173802 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0924 13:50:50.175162 (+ 1360us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 13:50:50.176390 (+ 1228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 13:50:50.176397 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 13:50:50.176400 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0924 13:50:50.176614 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 13:50:50.176805 (+ 191us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 13:50:50.176806 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 13:50:50.176806 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 13:50:50.176938 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 13:50:50.177166 (+ 228us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 13:50:50.177174 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 13:50:50.177323 (+ 149us) client_negotiation.cc:769] Sending connection context
0924 13:50:50.177434 (+ 111us) client_negotiation.cc:240] Negotiation successful
0924 13:50:50.177571 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":86,"threads_started":1}
I20260924 13:50:50.178741 30255 heartbeater.cc:347] Connected to a master server at 127.13.149.254:38197
I20260924 13:50:50.178843 30255 heartbeater.cc:464] Registering TS with master...
I20260924 13:50:50.179013 30255 heartbeater.cc:511] Master 127.13.149.254:38197 requested a full tablet report, sending...
I20260924 13:50:50.179454 29929 ts_manager.cc:194] Registered new tserver with Master: dff77ad6341d44f8a14ec7f008ba9d73 (127.13.149.194:37117)
I20260924 13:50:50.179822 29929 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.149.194@KRBTEST.COM'} at 127.13.149.194:56305
W20260924 13:50:50.257746 30258 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 13:50:50.257953 30258 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 13:50:50.257985 30258 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 13:50:50.259198 30258 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 13:50:50.259243 30258 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 13:50:50.259279 30258 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.149.195
I20260924 13:50:50.260782 30258 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.149.212:44125
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790257629041645-13911-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790257629041645-13911-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.195.keytab
--principal=kudu/127.13.149.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.149.195:42897
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790257629041645-13911-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.149.195
--webserver_port=43787
--tserver_master_addrs=127.13.149.254:38197
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30258
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.149.195
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790257629041645-13911-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 12:29:37 UTC on ccycloud.quasar-iqzbxn.root.comops.site
build id 4670106
I20260924 13:50:50.261065 30258 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 13:50:50.261309 30258 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 13:50:50.266466 30266 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 13:50:50.266486 30264 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 13:50:50.266474 30263 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 13:50:50.266700 30258 server_base.cc:1034] running on GCE node
I20260924 13:50:50.266884 30258 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 13:50:50.267119 30258 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 13:50:50.268289 30258 hybrid_clock.cc:648] HybridClock initialized: now 1790257850268269 us; error 29 us; skew 500 ppm
Sep 24 13:50:50 dist-test-slave-sjl2 krb5kdc[29348](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257850, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.149.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 13:50:50.270720 30258 init.cc:377] Logged in from keytab as kudu/127.13.149.195@KRBTEST.COM (short username kudu)
I20260924 13:50:50.271186 30258 webserver.cc:460] Webserver started at http://127.13.149.195:43787/ using document root <none> and password file <none>
I20260924 13:50:50.271392 30258 fs_manager.cc:359] Metadata directory not provided
I20260924 13:50:50.271454 30258 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 13:50:50.272804 30258 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260924 13:50:50.273399 30273 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 13:50:50.273558 30258 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260924 13:50:50.273648 30258 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790257629041645-13911-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790257629041645-13911-0/minicluster-data/ts-2/wal
uuid: "1c5172f7acb64efd980cb3d3e77d1851"
format_stamp: "Formatted at 2026-09-24 13:50:29 on dist-test-slave-sjl2"
server_key: "32f6b1efa03db5795adaa2278915a8ff"
server_key_iv: "cd51cfbb7b7a2aa88053a53cfe11ba87"
server_key_version: "encryptionkey@0"
I20260924 13:50:50.273983 30258 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790257629041645-13911-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790257629041645-13911-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790257629041645-13911-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 13:50:50.286512 30258 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 13:50:50.288362 30258 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 13:50:50.288507 30258 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 13:50:50.288723 30258 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 13:50:50.289042 30258 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 13:50:50.289083 30258 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 13:50:50.289113 30258 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 13:50:50.289137 30258 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 13:50:50.294655 30258 rpc_server.cc:307] RPC server started. Bound to: 127.13.149.195:42897
I20260924 13:50:50.294711 30386 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.149.195:42897 every 8 connection(s)
I20260924 13:50:50.295025 30258 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790257629041645-13911-0/minicluster-data/ts-2/data/info.pb
Sep 24 13:50:50 dist-test-slave-sjl2 krb5kdc[29348](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257850, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.149.195@KRBTEST.COM for kudu/127.13.149.254@KRBTEST.COM
I20260924 13:50:50.302269 13911 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJN9WRF/build/release/bin/kudu as pid 30258
I20260924 13:50:50.304235 30120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:50:50.295697 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.149.195:48183 (local address 127.13.149.254:38197)
0924 13:50:50.295750 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0924 13:50:50.295752 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:50:50.295939 (+ 187us) server_negotiation.cc:407] Connection header received
0924 13:50:50.296412 (+ 473us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:50:50.296413 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:50:50.296423 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:50:50.296473 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:50:50.297529 (+ 1056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:50:50.298075 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:50:50.299046 (+ 971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:50:50.299269 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:50:50.299328 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:50:50.301433 (+ 2105us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:50:50.301435 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:50:50.301435 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 13:50:50.301442 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:50:50.302854 (+ 1412us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:50:50.303266 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:50:50.303268 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:50:50.303268 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:50:50.303330 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:50:50.303539 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:50:50.303539 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:50:50.303540 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:50:50.303673 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:50:50.303722 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0924 13:50:50.304058 (+ 336us) server_negotiation.cc:299] Negotiation successful
0924 13:50:50.304164 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260924 13:50:50.304394 30389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:50:50.295685 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:38197 (local address 127.13.149.195:48183)
0924 13:50:50.295826 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0924 13:50:50.295861 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0924 13:50:50.296337 (+ 476us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:50:50.296577 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:50:50.296581 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:50:50.297249 (+ 668us) client_negotiation.cc:189] Negotiated authn=SASL
0924 13:50:50.297439 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:50:50.297444 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:50:50.298167 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:50:50.298168 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:50:50.298971 (+ 803us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:50:50.298973 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:50:50.299331 (+ 358us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:50:50.299331 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:50:50.299438 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:50:50.299945 (+ 507us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 13:50:50.299952 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0924 13:50:50.301349 (+ 1397us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 13:50:50.302925 (+ 1576us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 13:50:50.302932 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 13:50:50.302934 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 13:50:50.303198 (+ 264us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 13:50:50.303394 (+ 196us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 13:50:50.303395 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 13:50:50.303395 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 13:50:50.303486 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 13:50:50.303736 (+ 250us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 13:50:50.303742 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 13:50:50.303902 (+ 160us) client_negotiation.cc:769] Sending connection context
0924 13:50:50.304015 (+ 113us) client_negotiation.cc:240] Negotiation successful
0924 13:50:50.304171 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":76,"threads_started":1}
I20260924 13:50:50.305017 30387 heartbeater.cc:347] Connected to a master server at 127.13.149.254:38197
I20260924 13:50:50.305102 30387 heartbeater.cc:464] Registering TS with master...
I20260924 13:50:50.305305 30387 heartbeater.cc:511] Master 127.13.149.254:38197 requested a full tablet report, sending...
I20260924 13:50:50.305697 29929 ts_manager.cc:194] Registered new tserver with Master: 1c5172f7acb64efd980cb3d3e77d1851 (127.13.149.195:42897)
I20260924 13:50:50.306206 29929 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.149.195@KRBTEST.COM'} at 127.13.149.195:48183
I20260924 13:50:50.308950 30390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:50:50.302654 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41022 (local address 127.13.149.254:38197)
0924 13:50:50.302758 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0924 13:50:50.302760 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:50:50.302908 (+ 148us) server_negotiation.cc:407] Connection header received
0924 13:50:50.302966 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:50:50.302967 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:50:50.302978 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:50:50.303049 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:50:50.303638 (+ 589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:50:50.304341 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:50:50.305319 (+ 978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:50:50.305656 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:50:50.305719 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:50:50.306861 (+ 1142us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:50:50.306862 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:50:50.306863 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 13:50:50.306869 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:50:50.308144 (+ 1275us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:50:50.308420 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:50:50.308421 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:50:50.308421 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:50:50.308464 (+ 43us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:50:50.308570 (+ 106us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:50:50.308571 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:50:50.308571 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:50:50.308652 (+ 81us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:50:50.308771 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0924 13:50:50.308796 (+ 25us) server_negotiation.cc:299] Negotiation successful
0924 13:50:50.308853 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85,"thread_start_us":67,"threads_started":1}
I20260924 13:50:50.309653 13911 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260924 13:50:50.312409 30390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:50:50.309969 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41036 (local address 127.13.149.254:38197)
0924 13:50:50.310021 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0924 13:50:50.310023 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:50:50.310085 (+ 62us) server_negotiation.cc:407] Connection header received
0924 13:50:50.310166 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:50:50.310166 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:50:50.310199 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:50:50.310258 (+ 59us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 13:50:50.310386 (+ 128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:50:50.310846 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:50:50.311630 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:50:50.311844 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:50:50.311897 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:50:50.312041 (+ 144us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 13:50:50.312088 (+ 47us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0924 13:50:50.312343 (+ 255us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:41036: token expired
Metrics: {"server-negotiator.queue_time_us":31}
W20260924 13:50:50.312458 30390 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:41036: token expired
W20260924 13:50:50.312628 29858 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.13.149.254:38197)
I20260924 13:50:50.319203 30390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:50:50.312867 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41042 (local address 127.13.149.254:38197)
0924 13:50:50.312902 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0924 13:50:50.312903 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0924 13:50:50.312911 (+ 8us) server_negotiation.cc:407] Connection header received
0924 13:50:50.313011 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:50:50.313011 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:50:50.313020 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:50:50.313074 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:50:50.313635 (+ 561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:50:50.314315 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:50:50.315152 (+ 837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:50:50.315345 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:50:50.315398 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:50:50.316671 (+ 1273us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:50:50.316672 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:50:50.316672 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 13:50:50.316679 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:50:50.318106 (+ 1427us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:50:50.318420 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:50:50.318421 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:50:50.318421 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:50:50.318476 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:50:50.318639 (+ 163us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:50:50.318640 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:50:50.318640 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:50:50.318781 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:50:50.318871 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0924 13:50:50.319027 (+ 156us) server_negotiation.cc:299] Negotiation successful
0924 13:50:50.319115 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260924 13:50:50.320385 29929 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41042:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 13:50:50.322136 29929 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 13:50:50.324288 30051 tablet_service.cc:1467] Processing CreateTablet for tablet e6e9a3195d204d9495b51fa621106833 (DEFAULT_TABLE table=test-table [id=2f15459a46b845caa2f2e194fdb07c8e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 13:50:50.324450 30051 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e6e9a3195d204d9495b51fa621106833. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 13:50:50.325798 30009 tablet_bootstrap.cc:492] T e6e9a3195d204d9495b51fa621106833 P efba9ffdd4eb42929192092d77763fcc: Bootstrap starting.
I20260924 13:50:50.326115 30009 tablet_bootstrap.cc:654] T e6e9a3195d204d9495b51fa621106833 P efba9ffdd4eb42929192092d77763fcc: Neither blocks nor log segments found. Creating new log.
I20260924 13:50:50.326700 30009 tablet_bootstrap.cc:492] T e6e9a3195d204d9495b51fa621106833 P efba9ffdd4eb42929192092d77763fcc: No bootstrap required, opened a new log
I20260924 13:50:50.326753 30009 ts_tablet_manager.cc:1397] T e6e9a3195d204d9495b51fa621106833 P efba9ffdd4eb42929192092d77763fcc: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260924 13:50:50.326862 30009 raft_consensus.cc:348] T e6e9a3195d204d9495b51fa621106833 P efba9ffdd4eb42929192092d77763fcc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c5172f7acb64efd980cb3d3e77d1851" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 42897 } } peers { permanent_uuid: "dff77ad6341d44f8a14ec7f008ba9d73" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 37117 } } peers { permanent_uuid: "efba9ffdd4eb42929192092d77763fcc" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 35539 } }
I20260924 13:50:50.326933 30009 raft_consensus.cc:374] T e6e9a3195d204d9495b51fa621106833 P efba9ffdd4eb42929192092d77763fcc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 13:50:50.326977 30009 raft_consensus.cc:729] T e6e9a3195d204d9495b51fa621106833 P efba9ffdd4eb42929192092d77763fcc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efba9ffdd4eb42929192092d77763fcc, State: Initialized, Role: FOLLOWER
I20260924 13:50:50.327018 30009 consensus_queue.cc:260] T e6e9a3195d204d9495b51fa621106833 P efba9ffdd4eb42929192092d77763fcc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c5172f7acb64efd980cb3d3e77d1851" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 42897 } } peers { permanent_uuid: "dff77ad6341d44f8a14ec7f008ba9d73" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 37117 } } peers { permanent_uuid: "efba9ffdd4eb42929192092d77763fcc" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 35539 } }
I20260924 13:50:50.327090 30009 ts_tablet_manager.cc:1428] T e6e9a3195d204d9495b51fa621106833 P efba9ffdd4eb42929192092d77763fcc: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260924 13:50:50.327242 30124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:50:50.323841 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.194:37117 (local address 127.0.0.1:50580)
0924 13:50:50.323896 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0924 13:50:50.323908 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0924 13:50:50.323952 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:50:50.324284 (+ 332us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:50:50.324285 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:50:50.324286 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 13:50:50.324329 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:50:50.324330 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:50:50.325389 (+ 1059us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:50:50.325390 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:50:50.326184 (+ 794us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:50:50.326186 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:50:50.327013 (+ 827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:50:50.327014 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:50:50.327081 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:50:50.327085 (+ 4us) client_negotiation.cc:769] Sending connection context
0924 13:50:50.327111 (+ 26us) client_negotiation.cc:240] Negotiation successful
0924 13:50:50.327157 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35}
I20260924 13:50:50.327582 30394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:50:50.323930 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50580 (local address 127.13.149.194:37117)
0924 13:50:50.324086 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0924 13:50:50.324091 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 13:50:50.324100 (+ 9us) server_negotiation.cc:407] Connection header received
0924 13:50:50.324139 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:50:50.324141 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:50:50.324159 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:50:50.324273 (+ 114us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 13:50:50.324415 (+ 142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:50:50.325319 (+ 904us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:50:50.326258 (+ 939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:50:50.326920 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:50:50.327359 (+ 439us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:50:50.327432 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0924 13:50:50.327458 (+ 26us) server_negotiation.cc:299] Negotiation successful
0924 13:50:50.327501 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123,"thread_start_us":88,"threads_started":1}
I20260924 13:50:50.328188 30189 tablet_service.cc:1467] Processing CreateTablet for tablet e6e9a3195d204d9495b51fa621106833 (DEFAULT_TABLE table=test-table [id=2f15459a46b845caa2f2e194fdb07c8e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 13:50:50.328512 30189 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e6e9a3195d204d9495b51fa621106833. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 13:50:50.329298 30393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:50:50.323848 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.195:42897 (local address 127.0.0.1:36344)
0924 13:50:50.324142 (+ 294us) negotiation.cc:107] Waiting for socket to connect
0924 13:50:50.324151 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0924 13:50:50.324193 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:50:50.326134 (+ 1941us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:50:50.326135 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:50:50.326138 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 13:50:50.326187 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:50:50.326190 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:50:50.327359 (+ 1169us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:50:50.327359 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:50:50.328252 (+ 893us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:50:50.328255 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:50:50.329112 (+ 857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:50:50.329113 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:50:50.329160 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:50:50.329163 (+ 3us) client_negotiation.cc:769] Sending connection context
0924 13:50:50.329201 (+ 38us) client_negotiation.cc:240] Negotiation successful
0924 13:50:50.329233 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":177,"mutex_wait_us":22,"thread_start_us":100,"threads_started":1}
I20260924 13:50:50.329362 30395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:50:50.324105 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36344 (local address 127.13.149.195:42897)
0924 13:50:50.325880 (+ 1775us) server_negotiation.cc:206] Beginning negotiation
0924 13:50:50.325884 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 13:50:50.325895 (+ 11us) server_negotiation.cc:407] Connection header received
0924 13:50:50.325940 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:50:50.325942 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:50:50.325960 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:50:50.326038 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 13:50:50.326270 (+ 232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:50:50.327212 (+ 942us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:50:50.328336 (+ 1124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:50:50.329043 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:50:50.329114 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:50:50.329164 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0924 13:50:50.329223 (+ 59us) server_negotiation.cc:299] Negotiation successful
0924 13:50:50.329284 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1732,"thread_start_us":128,"threads_started":1}
I20260924 13:50:50.329926 30321 tablet_service.cc:1467] Processing CreateTablet for tablet e6e9a3195d204d9495b51fa621106833 (DEFAULT_TABLE table=test-table [id=2f15459a46b845caa2f2e194fdb07c8e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 13:50:50.330237 30321 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e6e9a3195d204d9495b51fa621106833. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 13:50:50.331583 30397 tablet_bootstrap.cc:492] T e6e9a3195d204d9495b51fa621106833 P dff77ad6341d44f8a14ec7f008ba9d73: Bootstrap starting.
I20260924 13:50:50.332316 30397 tablet_bootstrap.cc:654] T e6e9a3195d204d9495b51fa621106833 P dff77ad6341d44f8a14ec7f008ba9d73: Neither blocks nor log segments found. Creating new log.
I20260924 13:50:50.332903 30397 log.cc:824] T e6e9a3195d204d9495b51fa621106833 P dff77ad6341d44f8a14ec7f008ba9d73: Log is configured to *not* fsync() on all Append() calls
I20260924 13:50:50.333271 30398 tablet_bootstrap.cc:492] T e6e9a3195d204d9495b51fa621106833 P 1c5172f7acb64efd980cb3d3e77d1851: Bootstrap starting.
I20260924 13:50:50.333865 30397 tablet_bootstrap.cc:492] T e6e9a3195d204d9495b51fa621106833 P dff77ad6341d44f8a14ec7f008ba9d73: No bootstrap required, opened a new log
I20260924 13:50:50.333913 30398 tablet_bootstrap.cc:654] T e6e9a3195d204d9495b51fa621106833 P 1c5172f7acb64efd980cb3d3e77d1851: Neither blocks nor log segments found. Creating new log.
I20260924 13:50:50.333923 30397 ts_tablet_manager.cc:1397] T e6e9a3195d204d9495b51fa621106833 P dff77ad6341d44f8a14ec7f008ba9d73: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260924 13:50:50.334410 30398 log.cc:824] T e6e9a3195d204d9495b51fa621106833 P 1c5172f7acb64efd980cb3d3e77d1851: Log is configured to *not* fsync() on all Append() calls
I20260924 13:50:50.335062 30398 tablet_bootstrap.cc:492] T e6e9a3195d204d9495b51fa621106833 P 1c5172f7acb64efd980cb3d3e77d1851: No bootstrap required, opened a new log
I20260924 13:50:50.335121 30398 ts_tablet_manager.cc:1397] T e6e9a3195d204d9495b51fa621106833 P 1c5172f7acb64efd980cb3d3e77d1851: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260924 13:50:50.335511 30397 raft_consensus.cc:348] T e6e9a3195d204d9495b51fa621106833 P dff77ad6341d44f8a14ec7f008ba9d73 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c5172f7acb64efd980cb3d3e77d1851" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 42897 } } peers { permanent_uuid: "dff77ad6341d44f8a14ec7f008ba9d73" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 37117 } } peers { permanent_uuid: "efba9ffdd4eb42929192092d77763fcc" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 35539 } }
I20260924 13:50:50.335654 30397 raft_consensus.cc:374] T e6e9a3195d204d9495b51fa621106833 P dff77ad6341d44f8a14ec7f008ba9d73 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 13:50:50.335675 30397 raft_consensus.cc:729] T e6e9a3195d204d9495b51fa621106833 P dff77ad6341d44f8a14ec7f008ba9d73 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dff77ad6341d44f8a14ec7f008ba9d73, State: Initialized, Role: FOLLOWER
I20260924 13:50:50.335790 30397 consensus_queue.cc:260] T e6e9a3195d204d9495b51fa621106833 P dff77ad6341d44f8a14ec7f008ba9d73 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c5172f7acb64efd980cb3d3e77d1851" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 42897 } } peers { permanent_uuid: "dff77ad6341d44f8a14ec7f008ba9d73" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 37117 } } peers { permanent_uuid: "efba9ffdd4eb42929192092d77763fcc" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 35539 } }
I20260924 13:50:50.336071 30397 ts_tablet_manager.cc:1428] T e6e9a3195d204d9495b51fa621106833 P dff77ad6341d44f8a14ec7f008ba9d73: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260924 13:50:50.336186 30255 heartbeater.cc:503] Master 127.13.149.254:38197 was elected leader, sending a full tablet report...
I20260924 13:50:50.336835 30398 raft_consensus.cc:348] T e6e9a3195d204d9495b51fa621106833 P 1c5172f7acb64efd980cb3d3e77d1851 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c5172f7acb64efd980cb3d3e77d1851" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 42897 } } peers { permanent_uuid: "dff77ad6341d44f8a14ec7f008ba9d73" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 37117 } } peers { permanent_uuid: "efba9ffdd4eb42929192092d77763fcc" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 35539 } }
I20260924 13:50:50.337059 30398 raft_consensus.cc:374] T e6e9a3195d204d9495b51fa621106833 P 1c5172f7acb64efd980cb3d3e77d1851 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 13:50:50.337107 30398 raft_consensus.cc:729] T e6e9a3195d204d9495b51fa621106833 P 1c5172f7acb64efd980cb3d3e77d1851 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1c5172f7acb64efd980cb3d3e77d1851, State: Initialized, Role: FOLLOWER
I20260924 13:50:50.337224 30398 consensus_queue.cc:260] T e6e9a3195d204d9495b51fa621106833 P 1c5172f7acb64efd980cb3d3e77d1851 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c5172f7acb64efd980cb3d3e77d1851" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 42897 } } peers { permanent_uuid: "dff77ad6341d44f8a14ec7f008ba9d73" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 37117 } } peers { permanent_uuid: "efba9ffdd4eb42929192092d77763fcc" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 35539 } }
I20260924 13:50:50.337450 30398 ts_tablet_manager.cc:1428] T e6e9a3195d204d9495b51fa621106833 P 1c5172f7acb64efd980cb3d3e77d1851: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260924 13:50:50.337491 30387 heartbeater.cc:503] Master 127.13.149.254:38197 was elected leader, sending a full tablet report...
W20260924 13:50:50.419466 30256 tablet.cc:2369] T e6e9a3195d204d9495b51fa621106833 P dff77ad6341d44f8a14ec7f008ba9d73: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260924 13:50:50.529114 30402 raft_consensus.cc:482] T e6e9a3195d204d9495b51fa621106833 P 1c5172f7acb64efd980cb3d3e77d1851 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 13:50:50.529270 30402 raft_consensus.cc:504] T e6e9a3195d204d9495b51fa621106833 P 1c5172f7acb64efd980cb3d3e77d1851 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c5172f7acb64efd980cb3d3e77d1851" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 42897 } } peers { permanent_uuid: "dff77ad6341d44f8a14ec7f008ba9d73" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 37117 } } peers { permanent_uuid: "efba9ffdd4eb42929192092d77763fcc" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 35539 } }
I20260924 13:50:50.529587 30402 leader_election.cc:288] T e6e9a3195d204d9495b51fa621106833 P 1c5172f7acb64efd980cb3d3e77d1851 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers dff77ad6341d44f8a14ec7f008ba9d73 (127.13.149.194:37117), efba9ffdd4eb42929192092d77763fcc (127.13.149.193:35539)
I20260924 13:50:50.532660 30125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:50:50.529890 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.149.195:44589 (local address 127.13.149.193:35539)
0924 13:50:50.529930 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0924 13:50:50.529932 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:50:50.530100 (+ 168us) server_negotiation.cc:407] Connection header received
0924 13:50:50.530134 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:50:50.530135 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:50:50.530145 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:50:50.530199 (+ 54us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 13:50:50.530366 (+ 167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:50:50.531065 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:50:50.532005 (+ 940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:50:50.532361 (+ 356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:50:50.532422 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:50:50.532431 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0924 13:50:50.532555 (+ 124us) server_negotiation.cc:299] Negotiation successful
0924 13:50:50.532594 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260924 13:50:50.532675 30403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:50:50.529791 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.193:35539 (local address 127.13.149.195:44589)
0924 13:50:50.530029 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0924 13:50:50.530039 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 13:50:50.530093 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:50:50.530204 (+ 111us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:50:50.530205 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:50:50.530206 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 13:50:50.530278 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:50:50.530280 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:50:50.531138 (+ 858us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:50:50.531139 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:50:50.531922 (+ 783us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:50:50.531924 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:50:50.532430 (+ 506us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:50:50.532431 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:50:50.532496 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:50:50.532499 (+ 3us) client_negotiation.cc:769] Sending connection context
0924 13:50:50.532544 (+ 45us) client_negotiation.cc:240] Negotiation successful
0924 13:50:50.532583 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":220,"mutex_wait_us":66,"thread_start_us":100,"threads_started":1}
I20260924 13:50:50.532733 30394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:50:50.529859 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.149.195:47457 (local address 127.13.149.194:37117)
0924 13:50:50.529903 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0924 13:50:50.529905 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:50:50.529941 (+ 36us) server_negotiation.cc:407] Connection header received
0924 13:50:50.529971 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:50:50.529973 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:50:50.529983 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:50:50.530029 (+ 46us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 13:50:50.530204 (+ 175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:50:50.531002 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:50:50.532029 (+ 1027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:50:50.532428 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:50:50.532491 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:50:50.532499 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0924 13:50:50.532632 (+ 133us) server_negotiation.cc:299] Negotiation successful
0924 13:50:50.532670 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260924 13:50:50.532758 30389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:50:50.529786 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.194:37117 (local address 127.13.149.195:47457)
0924 13:50:50.529840 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0924 13:50:50.529852 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0924 13:50:50.529900 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:50:50.530050 (+ 150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:50:50.530051 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:50:50.530053 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 13:50:50.530136 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:50:50.530138 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:50:50.531107 (+ 969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:50:50.531108 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:50:50.531924 (+ 816us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:50:50.531926 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:50:50.532504 (+ 578us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:50:50.532505 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:50:50.532573 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:50:50.532576 (+ 3us) client_negotiation.cc:769] Sending connection context
0924 13:50:50.532621 (+ 45us) client_negotiation.cc:240] Negotiation successful
0924 13:50:50.532658 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30,"mutex_wait_us":58}
I20260924 13:50:50.533053 30209 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e6e9a3195d204d9495b51fa621106833" candidate_uuid: "1c5172f7acb64efd980cb3d3e77d1851" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dff77ad6341d44f8a14ec7f008ba9d73" is_pre_election: true
I20260924 13:50:50.533056 30071 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e6e9a3195d204d9495b51fa621106833" candidate_uuid: "1c5172f7acb64efd980cb3d3e77d1851" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efba9ffdd4eb42929192092d77763fcc" is_pre_election: true
I20260924 13:50:50.533195 30209 raft_consensus.cc:2454] T e6e9a3195d204d9495b51fa621106833 P dff77ad6341d44f8a14ec7f008ba9d73 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1c5172f7acb64efd980cb3d3e77d1851 in term 0.
I20260924 13:50:50.533210 30071 raft_consensus.cc:2454] T e6e9a3195d204d9495b51fa621106833 P efba9ffdd4eb42929192092d77763fcc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1c5172f7acb64efd980cb3d3e77d1851 in term 0.
I20260924 13:50:50.533450 30276 leader_election.cc:302] T e6e9a3195d204d9495b51fa621106833 P 1c5172f7acb64efd980cb3d3e77d1851 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1c5172f7acb64efd980cb3d3e77d1851, dff77ad6341d44f8a14ec7f008ba9d73; no voters:
I20260924 13:50:50.533576 30402 raft_consensus.cc:2781] T e6e9a3195d204d9495b51fa621106833 P 1c5172f7acb64efd980cb3d3e77d1851 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 13:50:50.533632 30402 raft_consensus.cc:482] T e6e9a3195d204d9495b51fa621106833 P 1c5172f7acb64efd980cb3d3e77d1851 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 13:50:50.533663 30402 raft_consensus.cc:3037] T e6e9a3195d204d9495b51fa621106833 P 1c5172f7acb64efd980cb3d3e77d1851 [term 0 FOLLOWER]: Advancing to term 1
I20260924 13:50:50.534302 30402 raft_consensus.cc:504] T e6e9a3195d204d9495b51fa621106833 P 1c5172f7acb64efd980cb3d3e77d1851 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c5172f7acb64efd980cb3d3e77d1851" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 42897 } } peers { permanent_uuid: "dff77ad6341d44f8a14ec7f008ba9d73" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 37117 } } peers { permanent_uuid: "efba9ffdd4eb42929192092d77763fcc" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 35539 } }
I20260924 13:50:50.534428 30402 leader_election.cc:288] T e6e9a3195d204d9495b51fa621106833 P 1c5172f7acb64efd980cb3d3e77d1851 [CANDIDATE]: Term 1 election: Requested vote from peers dff77ad6341d44f8a14ec7f008ba9d73 (127.13.149.194:37117), efba9ffdd4eb42929192092d77763fcc (127.13.149.193:35539)
W20260924 13:50:50.534462 30118 tablet.cc:2369] T e6e9a3195d204d9495b51fa621106833 P efba9ffdd4eb42929192092d77763fcc: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260924 13:50:50.534638 30071 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e6e9a3195d204d9495b51fa621106833" candidate_uuid: "1c5172f7acb64efd980cb3d3e77d1851" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efba9ffdd4eb42929192092d77763fcc"
I20260924 13:50:50.534651 30209 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e6e9a3195d204d9495b51fa621106833" candidate_uuid: "1c5172f7acb64efd980cb3d3e77d1851" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dff77ad6341d44f8a14ec7f008ba9d73"
I20260924 13:50:50.534708 30071 raft_consensus.cc:3037] T e6e9a3195d204d9495b51fa621106833 P efba9ffdd4eb42929192092d77763fcc [term 0 FOLLOWER]: Advancing to term 1
I20260924 13:50:50.534724 30209 raft_consensus.cc:3037] T e6e9a3195d204d9495b51fa621106833 P dff77ad6341d44f8a14ec7f008ba9d73 [term 0 FOLLOWER]: Advancing to term 1
I20260924 13:50:50.535408 30209 raft_consensus.cc:2454] T e6e9a3195d204d9495b51fa621106833 P dff77ad6341d44f8a14ec7f008ba9d73 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1c5172f7acb64efd980cb3d3e77d1851 in term 1.
I20260924 13:50:50.535584 30071 raft_consensus.cc:2454] T e6e9a3195d204d9495b51fa621106833 P efba9ffdd4eb42929192092d77763fcc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1c5172f7acb64efd980cb3d3e77d1851 in term 1.
I20260924 13:50:50.535598 30276 leader_election.cc:302] T e6e9a3195d204d9495b51fa621106833 P 1c5172f7acb64efd980cb3d3e77d1851 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1c5172f7acb64efd980cb3d3e77d1851, dff77ad6341d44f8a14ec7f008ba9d73; no voters:
I20260924 13:50:50.535746 30402 raft_consensus.cc:2781] T e6e9a3195d204d9495b51fa621106833 P 1c5172f7acb64efd980cb3d3e77d1851 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 13:50:50.535871 30402 raft_consensus.cc:686] T e6e9a3195d204d9495b51fa621106833 P 1c5172f7acb64efd980cb3d3e77d1851 [term 1 LEADER]: Becoming Leader. State: Replica: 1c5172f7acb64efd980cb3d3e77d1851, State: Running, Role: LEADER
I20260924 13:50:50.535949 30402 consensus_queue.cc:237] T e6e9a3195d204d9495b51fa621106833 P 1c5172f7acb64efd980cb3d3e77d1851 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c5172f7acb64efd980cb3d3e77d1851" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 42897 } } peers { permanent_uuid: "dff77ad6341d44f8a14ec7f008ba9d73" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 37117 } } peers { permanent_uuid: "efba9ffdd4eb42929192092d77763fcc" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 35539 } }
I20260924 13:50:50.536784 29928 catalog_manager.cc:5495] T e6e9a3195d204d9495b51fa621106833 P 1c5172f7acb64efd980cb3d3e77d1851 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1c5172f7acb64efd980cb3d3e77d1851 (127.13.149.195). New cstate: current_term: 1 leader_uuid: "1c5172f7acb64efd980cb3d3e77d1851" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c5172f7acb64efd980cb3d3e77d1851" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 42897 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dff77ad6341d44f8a14ec7f008ba9d73" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 37117 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efba9ffdd4eb42929192092d77763fcc" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 35539 } health_report { overall_health: UNKNOWN } } }
W20260924 13:50:50.545529 30388 tablet.cc:2369] T e6e9a3195d204d9495b51fa621106833 P 1c5172f7acb64efd980cb3d3e77d1851: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260924 13:50:50.558415 30395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:50:50.556300 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36356 (local address 127.13.149.195:42897)
0924 13:50:50.556348 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0924 13:50:50.556351 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 13:50:50.556358 (+ 7us) server_negotiation.cc:407] Connection header received
0924 13:50:50.556392 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:50:50.556393 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:50:50.556410 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:50:50.556479 (+ 69us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 13:50:50.556596 (+ 117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:50:50.557112 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:50:50.557805 (+ 693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:50:50.557984 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:50:50.558028 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:50:50.558127 (+ 99us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 13:50:50.558240 (+ 113us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 13:50:50.558300 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0924 13:50:50.558313 (+ 13us) server_negotiation.cc:299] Negotiation successful
0924 13:50:50.558332 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260924 13:50:50.560122 30209 raft_consensus.cc:1261] T e6e9a3195d204d9495b51fa621106833 P dff77ad6341d44f8a14ec7f008ba9d73 [term 1 FOLLOWER]: Refusing update from remote peer 1c5172f7acb64efd980cb3d3e77d1851: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 13:50:50.560130 30071 raft_consensus.cc:1261] T e6e9a3195d204d9495b51fa621106833 P efba9ffdd4eb42929192092d77763fcc [term 1 FOLLOWER]: Refusing update from remote peer 1c5172f7acb64efd980cb3d3e77d1851: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 13:50:50.560480 30404 consensus_queue.cc:1035] T e6e9a3195d204d9495b51fa621106833 P 1c5172f7acb64efd980cb3d3e77d1851 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dff77ad6341d44f8a14ec7f008ba9d73" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 37117 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 13:50:50.560606 30402 consensus_queue.cc:1035] T e6e9a3195d204d9495b51fa621106833 P 1c5172f7acb64efd980cb3d3e77d1851 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efba9ffdd4eb42929192092d77763fcc" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 35539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 13:50:50.562654 30406 mvcc.cc:205] Tried to move back new op lower bound from 7332896155891793920 to 7332896155795849216. Current Snapshot: MvccSnapshot[applied={T|T < 7332896155891793920}]
I20260924 13:50:50.562944 30407 mvcc.cc:205] Tried to move back new op lower bound from 7332896155891793920 to 7332896155795849216. Current Snapshot: MvccSnapshot[applied={T|T < 7332896155891793920}]
I20260924 13:50:50.563961 30409 mvcc.cc:205] Tried to move back new op lower bound from 7332896155891793920 to 7332896155795849216. Current Snapshot: MvccSnapshot[applied={T|T < 7332896155891793920 or (T in {7332896155891793920})}]
I20260924 13:50:50.565201 29928 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41042:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 13:50:50.565308 29928 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41042:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 13:50:50.566072 29928 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5d045e5a8ede44cba24125f78123692f: Sending DeleteTablet for 3 replicas of tablet e6e9a3195d204d9495b51fa621106833
I20260924 13:50:50.566404 30189 tablet_service.cc:1514] Processing DeleteTablet for tablet e6e9a3195d204d9495b51fa621106833 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 13:50:50 UTC) from {username='kudu', principal='kudu/127.13.149.254@KRBTEST.COM'} at 127.0.0.1:50580
I20260924 13:50:50.566443 13911 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJN9WRF/build/release/bin/kudu with pid 29967
I20260924 13:50:50.566497 30321 tablet_service.cc:1514] Processing DeleteTablet for tablet e6e9a3195d204d9495b51fa621106833 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 13:50:50 UTC) from {username='kudu', principal='kudu/127.13.149.254@KRBTEST.COM'} at 127.0.0.1:36344
I20260924 13:50:50.566605 30051 tablet_service.cc:1514] Processing DeleteTablet for tablet e6e9a3195d204d9495b51fa621106833 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 13:50:50 UTC) from {username='kudu', principal='kudu/127.13.149.254@KRBTEST.COM'} at 127.0.0.1:40462
I20260924 13:50:50.567073 30415 tablet_replica.cc:330] T e6e9a3195d204d9495b51fa621106833 P dff77ad6341d44f8a14ec7f008ba9d73: stopping tablet replica
I20260924 13:50:50.567171 30415 raft_consensus.cc:2229] T e6e9a3195d204d9495b51fa621106833 P dff77ad6341d44f8a14ec7f008ba9d73 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 13:50:50.567263 30415 raft_consensus.cc:2258] T e6e9a3195d204d9495b51fa621106833 P dff77ad6341d44f8a14ec7f008ba9d73 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 13:50:50.567718 30415 ts_tablet_manager.cc:1905] T e6e9a3195d204d9495b51fa621106833 P dff77ad6341d44f8a14ec7f008ba9d73: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 13:50:50.569044 30416 tablet_replica.cc:330] T e6e9a3195d204d9495b51fa621106833 P 1c5172f7acb64efd980cb3d3e77d1851: stopping tablet replica
I20260924 13:50:50.569170 30416 raft_consensus.cc:2229] T e6e9a3195d204d9495b51fa621106833 P 1c5172f7acb64efd980cb3d3e77d1851 [term 1 LEADER]: Raft consensus shutting down.
I20260924 13:50:50.569283 30416 raft_consensus.cc:2258] T e6e9a3195d204d9495b51fa621106833 P 1c5172f7acb64efd980cb3d3e77d1851 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 13:50:50.569677 30415 ts_tablet_manager.cc:1918] T e6e9a3195d204d9495b51fa621106833 P dff77ad6341d44f8a14ec7f008ba9d73: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 13:50:50.569751 30415 log.cc:1196] T e6e9a3195d204d9495b51fa621106833 P dff77ad6341d44f8a14ec7f008ba9d73: Deleting WAL directory at /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790257629041645-13911-0/minicluster-data/ts-1/wal/wals/e6e9a3195d204d9495b51fa621106833
I20260924 13:50:50.569741 30416 ts_tablet_manager.cc:1905] T e6e9a3195d204d9495b51fa621106833 P 1c5172f7acb64efd980cb3d3e77d1851: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 13:50:50.570003 30415 ts_tablet_manager.cc:1939] T e6e9a3195d204d9495b51fa621106833 P dff77ad6341d44f8a14ec7f008ba9d73: Deleting consensus metadata
I20260924 13:50:50.570533 29915 catalog_manager.cc:4841] TS dff77ad6341d44f8a14ec7f008ba9d73 (127.13.149.194:37117): tablet e6e9a3195d204d9495b51fa621106833 (table test-table [id=2f15459a46b845caa2f2e194fdb07c8e]) successfully deleted
I20260924 13:50:50.571290 30416 ts_tablet_manager.cc:1918] T e6e9a3195d204d9495b51fa621106833 P 1c5172f7acb64efd980cb3d3e77d1851: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 13:50:50.571363 30416 log.cc:1196] T e6e9a3195d204d9495b51fa621106833 P 1c5172f7acb64efd980cb3d3e77d1851: Deleting WAL directory at /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790257629041645-13911-0/minicluster-data/ts-2/wal/wals/e6e9a3195d204d9495b51fa621106833
I20260924 13:50:50.571626 30416 ts_tablet_manager.cc:1939] T e6e9a3195d204d9495b51fa621106833 P 1c5172f7acb64efd980cb3d3e77d1851: Deleting consensus metadata
W20260924 13:50:50.572000 29913 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.13.149.193:35539 (error 108)
I20260924 13:50:50.572209 29913 catalog_manager.cc:4841] TS 1c5172f7acb64efd980cb3d3e77d1851 (127.13.149.195:42897): tablet e6e9a3195d204d9495b51fa621106833 (table test-table [id=2f15459a46b845caa2f2e194fdb07c8e]) successfully deleted
I20260924 13:50:50.572387 13911 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJN9WRF/build/release/bin/kudu with pid 30122
I20260924 13:50:50.572479 30393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:50:50.572343 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.193:35539 (local address 127.0.0.1:40478)
0924 13:50:50.572402 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0924 13:50:50.572445 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.149.193:35539: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260924 13:50:50.573995 29913 catalog_manager.cc:4584] TS efba9ffdd4eb42929192092d77763fcc (127.13.149.193:35539): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet e6e9a3195d204d9495b51fa621106833: Network error: Client connection negotiation failed: client connection to 127.13.149.193:35539: connect: Connection refused (error 111)
I20260924 13:50:50.577292 13911 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJN9WRF/build/release/bin/kudu with pid 30258
I20260924 13:50:50.583771 13911 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJN9WRF/build/release/bin/kudu with pid 29896
I20260924 13:51:10.589035 13911 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJN9WRF/build/release/bin/kudu
/tmp/dist-test-taskJN9WRF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790257629041645-13911-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790257629041645-13911-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790257629041645-13911-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790257629041645-13911-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.149.254:38197
--webserver_interface=127.13.149.254
--webserver_port=34079
--keytab_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.254.keytab
--principal=kudu/127.13.149.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.149.212:44125
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.149.254:38197
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790257629041645-13911-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790257629041645-13911-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790257629041645-13911-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 13:51:10.669884 30418 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 13:51:10.670094 30418 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 13:51:10.670115 30418 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 13:51:10.671260 30418 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260924 13:51:10.671300 30418 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260924 13:51:10.671311 30418 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260924 13:51:10.671321 30418 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 13:51:10.671334 30418 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 13:51:10.671345 30418 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 13:51:10.671368 30418 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 13:51:10.671391 30418 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 13:51:10.672871 30418 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.149.212:44125
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790257629041645-13911-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790257629041645-13911-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.13.149.254:38197
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.254.keytab
--principal=kudu/127.13.149.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.149.254:38197
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790257629041645-13911-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.149.254
--webserver_port=34079
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30418
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790257629041645-13911-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 12:29:37 UTC on ccycloud.quasar-iqzbxn.root.comops.site
build id 4670106
I20260924 13:51:10.673151 30418 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 13:51:10.673369 30418 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260924 13:51:10.678505 30418 server_base.cc:1034] running on GCE node
W20260924 13:51:10.678488 30423 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 13:51:10.678488 30426 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 13:51:10.678486 30424 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 13:51:10.679018 30418 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 13:51:10.679312 30418 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 13:51:10.680462 30418 hybrid_clock.cc:648] HybridClock initialized: now 1790257870680449 us; error 31 us; skew 500 ppm
Sep 24 13:51:10 dist-test-slave-sjl2 krb5kdc[29348](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257870, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.149.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 13:51:10.689435 30418 init.cc:377] Logged in from keytab as kudu/127.13.149.254@KRBTEST.COM (short username kudu)
I20260924 13:51:10.689864 30418 webserver.cc:460] Webserver started at http://127.13.149.254:34079/ using document root <none> and password file <none>
I20260924 13:51:10.690052 30418 fs_manager.cc:359] Metadata directory not provided
I20260924 13:51:10.690091 30418 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 13:51:10.691367 30418 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260924 13:51:10.691963 30433 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 13:51:10.692122 30418 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 13:51:10.692230 30418 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790257629041645-13911-0/minicluster-data/master-0/data,/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790257629041645-13911-0/minicluster-data/master-0/wal
uuid: "5d045e5a8ede44cba24125f78123692f"
format_stamp: "Formatted at 2026-09-24 13:50:28 on dist-test-slave-sjl2"
server_key: "cb869e500d58c01ee6af39032eb28186"
server_key_iv: "3152d23837859a2898ff482ee43ee262"
server_key_version: "encryptionkey@0"
I20260924 13:51:10.692564 30418 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790257629041645-13911-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790257629041645-13911-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790257629041645-13911-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 13:51:10.705971 30418 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 13:51:10.708057 30418 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 13:51:10.708201 30418 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 13:51:10.711819 30418 rpc_server.cc:307] RPC server started. Bound to: 127.13.149.254:38197
I20260924 13:51:10.711853 30485 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.149.254:38197 every 8 connection(s)
I20260924 13:51:10.712203 30418 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790257629041645-13911-0/minicluster-data/master-0/data/info.pb
I20260924 13:51:10.713686 30486 sys_catalog.cc:263] Verifying existing consensus state
I20260924 13:51:10.713956 13911 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJN9WRF/build/release/bin/kudu as pid 30418
I20260924 13:51:10.714236 13911 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJN9WRF/build/release/bin/kudu
/tmp/dist-test-taskJN9WRF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790257629041645-13911-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790257629041645-13911-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790257629041645-13911-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790257629041645-13911-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.149.193:35539
--local_ip_for_outbound_sockets=127.13.149.193
--tserver_master_addrs=127.13.149.254:38197
--webserver_port=46623
--webserver_interface=127.13.149.193
--keytab_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.193.keytab
--principal=kudu/127.13.149.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.149.212:44125
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790257629041645-13911-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790257629041645-13911-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790257629041645-13911-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260924 13:51:10.714529 30486 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5d045e5a8ede44cba24125f78123692f: Bootstrap starting.
I20260924 13:51:10.723290 30486 log.cc:824] T 00000000000000000000000000000000 P 5d045e5a8ede44cba24125f78123692f: Log is configured to *not* fsync() on all Append() calls
I20260924 13:51:10.725569 30486 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5d045e5a8ede44cba24125f78123692f: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260924 13:51:10.725845 30486 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5d045e5a8ede44cba24125f78123692f: Bootstrap complete.
I20260924 13:51:10.728001 30486 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5d045e5a8ede44cba24125f78123692f [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d045e5a8ede44cba24125f78123692f" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 38197 } }
I20260924 13:51:10.728225 30486 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5d045e5a8ede44cba24125f78123692f [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5d045e5a8ede44cba24125f78123692f, State: Initialized, Role: FOLLOWER
I20260924 13:51:10.728400 30486 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5d045e5a8ede44cba24125f78123692f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d045e5a8ede44cba24125f78123692f" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 38197 } }
I20260924 13:51:10.728478 30486 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5d045e5a8ede44cba24125f78123692f [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 13:51:10.728528 30486 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5d045e5a8ede44cba24125f78123692f [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 13:51:10.728574 30486 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5d045e5a8ede44cba24125f78123692f [term 2 FOLLOWER]: Advancing to term 3
I20260924 13:51:10.729605 30486 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5d045e5a8ede44cba24125f78123692f [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d045e5a8ede44cba24125f78123692f" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 38197 } }
I20260924 13:51:10.729701 30486 leader_election.cc:302] T 00000000000000000000000000000000 P 5d045e5a8ede44cba24125f78123692f [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5d045e5a8ede44cba24125f78123692f; no voters:
I20260924 13:51:10.729866 30486 leader_election.cc:288] T 00000000000000000000000000000000 P 5d045e5a8ede44cba24125f78123692f [CANDIDATE]: Term 3 election: Requested vote from peers
I20260924 13:51:10.729918 30490 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5d045e5a8ede44cba24125f78123692f [term 3 FOLLOWER]: Leader election won for term 3
I20260924 13:51:10.730036 30490 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5d045e5a8ede44cba24125f78123692f [term 3 LEADER]: Becoming Leader. State: Replica: 5d045e5a8ede44cba24125f78123692f, State: Running, Role: LEADER
I20260924 13:51:10.730130 30486 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5d045e5a8ede44cba24125f78123692f [sys.catalog]: configured and running, proceeding with master startup.
I20260924 13:51:10.730122 30490 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5d045e5a8ede44cba24125f78123692f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d045e5a8ede44cba24125f78123692f" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 38197 } }
I20260924 13:51:10.730392 30491 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5d045e5a8ede44cba24125f78123692f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "5d045e5a8ede44cba24125f78123692f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d045e5a8ede44cba24125f78123692f" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 38197 } } }
I20260924 13:51:10.730400 30492 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5d045e5a8ede44cba24125f78123692f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5d045e5a8ede44cba24125f78123692f. Latest consensus state: current_term: 3 leader_uuid: "5d045e5a8ede44cba24125f78123692f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d045e5a8ede44cba24125f78123692f" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 38197 } } }
I20260924 13:51:10.730490 30491 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5d045e5a8ede44cba24125f78123692f [sys.catalog]: This master's current role is: LEADER
I20260924 13:51:10.730499 30492 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5d045e5a8ede44cba24125f78123692f [sys.catalog]: This master's current role is: LEADER
I20260924 13:51:10.731355 30495 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 13:51:10.732142 30495 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 13:51:10.732241 30495 catalog_manager.cc:1259] Loaded cluster ID: 6d304b4b2c5c4dfc889135d5e32001ae
I20260924 13:51:10.732264 30495 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 13:51:10.733441 30495 catalog_manager.cc:1504] Loading token signing keys...
I20260924 13:51:10.733609 30495 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 5d045e5a8ede44cba24125f78123692f: Loaded TSK: 0
I20260924 13:51:10.733640 30495 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 5d045e5a8ede44cba24125f78123692f: Loaded TSK: 1
F20260924 13:51:10.733865 30495 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f6bfaada9ad google::LogMessage::Fail() at ??:0
@ 0x7f6bfaadd88a google::LogMessage::SendToLog() at ??:0
@ 0x7f6bfaada4de google::LogMessage::Flush() at ??:0
@ 0x7f6bfaadba19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f6bfd247104 kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7f6bfd244d41 kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7f6c02a341cd kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7f6c02a3448e kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7f6c02a34594 _ZNSt17_Function_handlerIFN4kudu6StatusEvEZNS0_6master14CatalogManager24PrepareForLeadershipTaskEvEUlvE3_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x7f6c029fac48 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7f6c02a04ad0 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f6bfc9a80d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f6bfc99d954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f6bfd8016db start_thread at ??:0
@ 0x7f6bf95d861f clone at ??:0
W20260924 13:51:10.812175 30488 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 13:51:10.812382 30488 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 13:51:10.812413 30488 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 13:51:10.813630 30488 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 13:51:10.813683 30488 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 13:51:10.813722 30488 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.149.193
I20260924 13:51:10.815192 30488 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.149.212:44125
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790257629041645-13911-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790257629041645-13911-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.193.keytab
--principal=kudu/127.13.149.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.149.193:35539
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790257629041645-13911-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.149.193
--webserver_port=46623
--tserver_master_addrs=127.13.149.254:38197
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30488
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.149.193
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790257629041645-13911-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 12:29:37 UTC on ccycloud.quasar-iqzbxn.root.comops.site
build id 4670106
I20260924 13:51:10.815438 30488 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 13:51:10.815646 30488 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 13:51:10.820636 30528 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 13:51:10.820749 30488 server_base.cc:1034] running on GCE node
W20260924 13:51:10.820667 30527 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 13:51:10.820789 30530 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 13:51:10.821012 30488 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 13:51:10.821292 30488 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 13:51:10.822435 30488 hybrid_clock.cc:648] HybridClock initialized: now 1790257870822422 us; error 31 us; skew 500 ppm
Sep 24 13:51:10 dist-test-slave-sjl2 krb5kdc[29348](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257870, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.149.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 13:51:10.824774 30488 init.cc:377] Logged in from keytab as kudu/127.13.149.193@KRBTEST.COM (short username kudu)
I20260924 13:51:10.825232 30488 webserver.cc:460] Webserver started at http://127.13.149.193:46623/ using document root <none> and password file <none>
I20260924 13:51:10.825456 30488 fs_manager.cc:359] Metadata directory not provided
I20260924 13:51:10.825501 30488 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 13:51:10.826789 30488 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260924 13:51:10.827440 30537 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 13:51:10.827636 30488 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260924 13:51:10.827723 30488 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790257629041645-13911-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790257629041645-13911-0/minicluster-data/ts-0/wal
uuid: "efba9ffdd4eb42929192092d77763fcc"
format_stamp: "Formatted at 2026-09-24 13:50:29 on dist-test-slave-sjl2"
server_key: "eb4a105b307721dfe0e8a662ed190d66"
server_key_iv: "418a798cc85a0cd4db844acb8c5ff085"
server_key_version: "encryptionkey@0"
I20260924 13:51:10.828066 30488 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790257629041645-13911-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790257629041645-13911-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790257629041645-13911-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 13:51:10.849663 30488 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 13:51:10.851962 30488 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 13:51:10.852129 30488 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 13:51:10.852370 30488 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 13:51:10.852834 30544 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260924 13:51:10.854846 30488 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260924 13:51:10.854902 30488 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260924 13:51:10.854939 30488 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260924 13:51:10.855568 30488 ts_tablet_manager.cc:610] Registered 1 tablets
I20260924 13:51:10.855600 30488 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260924 13:51:10.855706 30544 tablet_bootstrap.cc:492] T e6e9a3195d204d9495b51fa621106833 P efba9ffdd4eb42929192092d77763fcc: Bootstrap starting.
I20260924 13:51:10.862232 30488 rpc_server.cc:307] RPC server started. Bound to: 127.13.149.193:35539
I20260924 13:51:10.862411 30651 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.149.193:35539 every 8 connection(s)
I20260924 13:51:10.862687 30488 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790257629041645-13911-0/minicluster-data/ts-0/data/info.pb
I20260924 13:51:10.867425 30544 log.cc:824] T e6e9a3195d204d9495b51fa621106833 P efba9ffdd4eb42929192092d77763fcc: Log is configured to *not* fsync() on all Append() calls
I20260924 13:51:10.869680 13911 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJN9WRF/build/release/bin/kudu as pid 30488
I20260924 13:51:10.869948 13911 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJN9WRF/build/release/bin/kudu
/tmp/dist-test-taskJN9WRF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790257629041645-13911-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790257629041645-13911-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790257629041645-13911-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790257629041645-13911-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.149.194:37117
--local_ip_for_outbound_sockets=127.13.149.194
--tserver_master_addrs=127.13.149.254:38197
--webserver_port=32807
--webserver_interface=127.13.149.194
--keytab_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.194.keytab
--principal=kudu/127.13.149.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.149.212:44125
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790257629041645-13911-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790257629041645-13911-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790257629041645-13911-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260924 13:51:10.870488 30544 tablet_bootstrap.cc:492] T e6e9a3195d204d9495b51fa621106833 P efba9ffdd4eb42929192092d77763fcc: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260924 13:51:10.870802 30544 tablet_bootstrap.cc:492] T e6e9a3195d204d9495b51fa621106833 P efba9ffdd4eb42929192092d77763fcc: Bootstrap complete.
I20260924 13:51:10.871943 30544 ts_tablet_manager.cc:1397] T e6e9a3195d204d9495b51fa621106833 P efba9ffdd4eb42929192092d77763fcc: Time spent bootstrapping tablet: real 0.016s user 0.012s sys 0.004s
I20260924 13:51:10.873544 30544 raft_consensus.cc:348] T e6e9a3195d204d9495b51fa621106833 P efba9ffdd4eb42929192092d77763fcc [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c5172f7acb64efd980cb3d3e77d1851" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 42897 } } peers { permanent_uuid: "dff77ad6341d44f8a14ec7f008ba9d73" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 37117 } } peers { permanent_uuid: "efba9ffdd4eb42929192092d77763fcc" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 35539 } }
I20260924 13:51:10.873893 30544 raft_consensus.cc:729] T e6e9a3195d204d9495b51fa621106833 P efba9ffdd4eb42929192092d77763fcc [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: efba9ffdd4eb42929192092d77763fcc, State: Initialized, Role: FOLLOWER
I20260924 13:51:10.874064 30544 consensus_queue.cc:260] T e6e9a3195d204d9495b51fa621106833 P efba9ffdd4eb42929192092d77763fcc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c5172f7acb64efd980cb3d3e77d1851" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 42897 } } peers { permanent_uuid: "dff77ad6341d44f8a14ec7f008ba9d73" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 37117 } } peers { permanent_uuid: "efba9ffdd4eb42929192092d77763fcc" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 35539 } }
I20260924 13:51:10.874372 30544 ts_tablet_manager.cc:1428] T e6e9a3195d204d9495b51fa621106833 P efba9ffdd4eb42929192092d77763fcc: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
W20260924 13:51:10.933792 30654 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 13:51:10.863401 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:38197 (local address 127.13.149.193:38187)
0924 13:51:10.863638 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0924 13:51:10.863683 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0924 13:51:10.864203 (+ 520us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:51:10.933581 (+ 69378us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.149.254:38197: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":111,"threads_started":1}
W20260924 13:51:10.934057 30652 heartbeater.cc:644] Failed to heartbeat to 127.13.149.254:38197 (0 consecutive failures): Network error: Failed to ping master at 127.13.149.254:38197: Client connection negotiation failed: client connection to 127.13.149.254:38197: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260924 13:51:10.934398 30654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:10.934290 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:38197 (local address 127.13.149.193:41843)
0924 13:51:10.934337 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0924 13:51:10.934375 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.149.254:38197: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
I20260924 13:51:10.934764 30654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:10.934671 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:38197 (local address 127.13.149.193:40343)
0924 13:51:10.934728 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0924 13:51:10.934749 (+ 21us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.149.254:38197: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260924 13:51:10.934870 30652 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260924 13:51:10.952369 30656 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 13:51:10.952600 30656 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 13:51:10.952625 30656 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 13:51:10.953857 30656 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 13:51:10.953907 30656 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 13:51:10.953933 30656 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.149.194
I20260924 13:51:10.955435 30656 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.149.212:44125
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790257629041645-13911-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790257629041645-13911-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.194.keytab
--principal=kudu/127.13.149.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.149.194:37117
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790257629041645-13911-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.149.194
--webserver_port=32807
--tserver_master_addrs=127.13.149.254:38197
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30656
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.149.194
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790257629041645-13911-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 12:29:37 UTC on ccycloud.quasar-iqzbxn.root.comops.site
build id 4670106
I20260924 13:51:10.955662 30656 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 13:51:10.955850 30656 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 13:51:10.960865 30663 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 13:51:10.960843 30662 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 13:51:10.960899 30656 server_base.cc:1034] running on GCE node
W20260924 13:51:10.960848 30665 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 13:51:10.961232 30656 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 13:51:10.961472 30656 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 13:51:10.962652 30656 hybrid_clock.cc:648] HybridClock initialized: now 1790257870962633 us; error 28 us; skew 500 ppm
Sep 24 13:51:10 dist-test-slave-sjl2 krb5kdc[29348](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257870, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.149.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 13:51:10.965258 30656 init.cc:377] Logged in from keytab as kudu/127.13.149.194@KRBTEST.COM (short username kudu)
I20260924 13:51:10.965766 30656 webserver.cc:460] Webserver started at http://127.13.149.194:32807/ using document root <none> and password file <none>
I20260924 13:51:10.965988 30656 fs_manager.cc:359] Metadata directory not provided
I20260924 13:51:10.966037 30656 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 13:51:10.967376 30656 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260924 13:51:10.967967 30672 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 13:51:10.968139 30656 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260924 13:51:10.968228 30656 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790257629041645-13911-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790257629041645-13911-0/minicluster-data/ts-1/wal
uuid: "dff77ad6341d44f8a14ec7f008ba9d73"
format_stamp: "Formatted at 2026-09-24 13:50:29 on dist-test-slave-sjl2"
server_key: "f20e10ce3675c422a53ee4a58a8351a5"
server_key_iv: "46011d7719e88455d00b6bad86f4363d"
server_key_version: "encryptionkey@0"
I20260924 13:51:10.968564 30656 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790257629041645-13911-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790257629041645-13911-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790257629041645-13911-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 13:51:10.979139 30656 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 13:51:10.981045 30656 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 13:51:10.981192 30656 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 13:51:10.981400 30656 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 13:51:10.981686 30656 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 13:51:10.981719 30656 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 13:51:10.981748 30656 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 13:51:10.981772 30656 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 13:51:10.987273 30656 rpc_server.cc:307] RPC server started. Bound to: 127.13.149.194:37117
I20260924 13:51:10.987327 30785 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.149.194:37117 every 8 connection(s)
I20260924 13:51:10.987650 30656 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790257629041645-13911-0/minicluster-data/ts-1/data/info.pb
I20260924 13:51:10.988612 30788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:10.988258 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:38197 (local address 127.13.149.194:55701)
0924 13:51:10.988406 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0924 13:51:10.988476 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.149.254:38197: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":79,"threads_started":1}
W20260924 13:51:10.989046 30786 heartbeater.cc:644] Failed to heartbeat to 127.13.149.254:38197 (0 consecutive failures): Network error: Failed to ping master at 127.13.149.254:38197: Client connection negotiation failed: client connection to 127.13.149.254:38197: connect: Connection refused (error 111)
I20260924 13:51:10.989358 30788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:10.989263 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:38197 (local address 127.13.149.194:41191)
0924 13:51:10.989309 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0924 13:51:10.989334 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.149.254:38197: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
I20260924 13:51:10.989596 30788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:10.989522 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:38197 (local address 127.13.149.194:37009)
0924 13:51:10.989557 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0924 13:51:10.989577 (+ 20us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.149.254:38197: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
W20260924 13:51:10.989692 30786 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260924 13:51:10.996069 13911 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJN9WRF/build/release/bin/kudu as pid 30656
I20260924 13:51:10.996318 13911 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJN9WRF/build/release/bin/kudu
/tmp/dist-test-taskJN9WRF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790257629041645-13911-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790257629041645-13911-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790257629041645-13911-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790257629041645-13911-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.149.195:42897
--local_ip_for_outbound_sockets=127.13.149.195
--tserver_master_addrs=127.13.149.254:38197
--webserver_port=43787
--webserver_interface=127.13.149.195
--keytab_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.195.keytab
--principal=kudu/127.13.149.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.149.212:44125
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790257629041645-13911-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790257629041645-13911-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790257629041645-13911-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 13:51:11.077740 30789 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 13:51:11.077967 30789 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 13:51:11.077992 30789 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 13:51:11.079119 30789 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 13:51:11.079161 30789 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 13:51:11.079190 30789 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.149.195
I20260924 13:51:11.080729 30789 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.149.212:44125
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790257629041645-13911-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790257629041645-13911-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.195.keytab
--principal=kudu/127.13.149.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.149.195:42897
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790257629041645-13911-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.149.195
--webserver_port=43787
--tserver_master_addrs=127.13.149.254:38197
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30789
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.149.195
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790257629041645-13911-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 12:29:37 UTC on ccycloud.quasar-iqzbxn.root.comops.site
build id 4670106
I20260924 13:51:11.080987 30789 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 13:51:11.081205 30789 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 13:51:11.086259 30794 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 13:51:11.086313 30797 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 13:51:11.086373 30789 server_base.cc:1034] running on GCE node
W20260924 13:51:11.086308 30795 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 13:51:11.086741 30789 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 13:51:11.086958 30789 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 13:51:11.088158 30789 hybrid_clock.cc:648] HybridClock initialized: now 1790257871088135 us; error 37 us; skew 500 ppm
Sep 24 13:51:11 dist-test-slave-sjl2 krb5kdc[29348](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257871, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.149.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 13:51:11.090648 30789 init.cc:377] Logged in from keytab as kudu/127.13.149.195@KRBTEST.COM (short username kudu)
I20260924 13:51:11.091068 30789 webserver.cc:460] Webserver started at http://127.13.149.195:43787/ using document root <none> and password file <none>
I20260924 13:51:11.091297 30789 fs_manager.cc:359] Metadata directory not provided
I20260924 13:51:11.091343 30789 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 13:51:11.092638 30789 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260924 13:51:11.093328 30804 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 13:51:11.093488 30789 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260924 13:51:11.093568 30789 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790257629041645-13911-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790257629041645-13911-0/minicluster-data/ts-2/wal
uuid: "1c5172f7acb64efd980cb3d3e77d1851"
format_stamp: "Formatted at 2026-09-24 13:50:29 on dist-test-slave-sjl2"
server_key: "32f6b1efa03db5795adaa2278915a8ff"
server_key_iv: "cd51cfbb7b7a2aa88053a53cfe11ba87"
server_key_version: "encryptionkey@0"
I20260924 13:51:11.093914 30789 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790257629041645-13911-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790257629041645-13911-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790257629041645-13911-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 13:51:11.109661 30789 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 13:51:11.111577 30789 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 13:51:11.111685 30789 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 13:51:11.111874 30789 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 13:51:11.112167 30789 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 13:51:11.112200 30789 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 13:51:11.112223 30789 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 13:51:11.112246 30789 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 13:51:11.117581 30789 rpc_server.cc:307] RPC server started. Bound to: 127.13.149.195:42897
I20260924 13:51:11.117623 30917 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.149.195:42897 every 8 connection(s)
I20260924 13:51:11.117969 30789 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790257629041645-13911-0/minicluster-data/ts-2/data/info.pb
I20260924 13:51:11.118964 30920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:11.118596 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:38197 (local address 127.13.149.195:52465)
0924 13:51:11.118756 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0924 13:51:11.118829 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.149.254:38197: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":65,"threads_started":1}
W20260924 13:51:11.119350 30918 heartbeater.cc:644] Failed to heartbeat to 127.13.149.254:38197 (0 consecutive failures): Network error: Failed to ping master at 127.13.149.254:38197: Client connection negotiation failed: client connection to 127.13.149.254:38197: connect: Connection refused (error 111)
I20260924 13:51:11.119601 30920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:11.119510 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:38197 (local address 127.13.149.195:45247)
0924 13:51:11.119559 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0924 13:51:11.119585 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.149.254:38197: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260924 13:51:11.119825 30920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:11.119766 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:38197 (local address 127.13.149.195:40085)
0924 13:51:11.119796 (+ 30us) negotiation.cc:107] Waiting for socket to connect
0924 13:51:11.119814 (+ 18us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.149.254:38197: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":21}
W20260924 13:51:11.119908 30918 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260924 13:51:11.121598 13911 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJN9WRF/build/release/bin/kudu as pid 30789
I20260924 13:51:11.935606 30922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:11.935348 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:38197 (local address 127.13.149.193:60137)
0924 13:51:11.935501 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0924 13:51:11.935541 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.149.254:38197: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":67,"threads_started":1}
I20260924 13:51:11.990367 30923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:11.990103 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:38197 (local address 127.13.149.194:40933)
0924 13:51:11.990274 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0924 13:51:11.990309 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.149.254:38197: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":71,"threads_started":1}
I20260924 13:51:12.120468 30924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:12.120251 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:38197 (local address 127.13.149.195:57011)
0924 13:51:12.120374 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0924 13:51:12.120414 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.149.254:38197: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":61,"threads_started":1}
I20260924 13:51:12.206883 30925 raft_consensus.cc:482] T e6e9a3195d204d9495b51fa621106833 P efba9ffdd4eb42929192092d77763fcc [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 13:51:12.207006 30925 raft_consensus.cc:504] T e6e9a3195d204d9495b51fa621106833 P efba9ffdd4eb42929192092d77763fcc [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c5172f7acb64efd980cb3d3e77d1851" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 42897 } } peers { permanent_uuid: "dff77ad6341d44f8a14ec7f008ba9d73" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 37117 } } peers { permanent_uuid: "efba9ffdd4eb42929192092d77763fcc" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 35539 } }
I20260924 13:51:12.207307 30925 leader_election.cc:288] T e6e9a3195d204d9495b51fa621106833 P efba9ffdd4eb42929192092d77763fcc [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1c5172f7acb64efd980cb3d3e77d1851 (127.13.149.195:42897), dff77ad6341d44f8a14ec7f008ba9d73 (127.13.149.194:37117)
Sep 24 13:51:12 dist-test-slave-sjl2 krb5kdc[29348](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257870, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.149.193@KRBTEST.COM for kudu/127.13.149.194@KRBTEST.COM
Sep 24 13:51:12 dist-test-slave-sjl2 krb5kdc[29348](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257870, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.149.193@KRBTEST.COM for kudu/127.13.149.195@KRBTEST.COM
I20260924 13:51:12.217368 30927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:12.207602 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.149.193:41875 (local address 127.13.149.195:42897)
0924 13:51:12.207775 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0924 13:51:12.207787 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0924 13:51:12.207822 (+ 35us) server_negotiation.cc:407] Connection header received
0924 13:51:12.208305 (+ 483us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:51:12.208308 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:51:12.208348 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:51:12.208471 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:51:12.209224 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:51:12.210079 (+ 855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:51:12.211030 (+ 951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:51:12.211434 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:51:12.211532 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:51:12.214130 (+ 2598us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:51:12.214134 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:51:12.214134 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 13:51:12.214144 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:51:12.215826 (+ 1682us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:51:12.216226 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:51:12.216229 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:51:12.216229 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:51:12.216325 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:51:12.216527 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:51:12.216528 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:51:12.216529 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:51:12.216762 (+ 233us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:51:12.216826 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0924 13:51:12.217134 (+ 308us) server_negotiation.cc:299] Negotiation successful
0924 13:51:12.217258 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":140,"thread_start_us":59,"threads_started":1}
I20260924 13:51:12.217373 30928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:12.207682 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.149.193:49551 (local address 127.13.149.194:37117)
0924 13:51:12.207825 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0924 13:51:12.207836 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0924 13:51:12.207870 (+ 34us) server_negotiation.cc:407] Connection header received
0924 13:51:12.208291 (+ 421us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:51:12.208293 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:51:12.208332 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:51:12.208449 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:51:12.209231 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:51:12.210073 (+ 842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:51:12.211031 (+ 958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:51:12.211421 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:51:12.211507 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:51:12.214113 (+ 2606us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:51:12.214121 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:51:12.214121 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 13:51:12.214131 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:51:12.215722 (+ 1591us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:51:12.216192 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:51:12.216196 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:51:12.216196 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:51:12.216292 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:51:12.216530 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:51:12.216531 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:51:12.216531 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:51:12.216762 (+ 231us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:51:12.216853 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0924 13:51:12.217135 (+ 282us) server_negotiation.cc:299] Negotiation successful
0924 13:51:12.217253 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":110,"thread_start_us":57,"threads_started":1}
I20260924 13:51:12.217391 30922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:12.207497 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.195:42897 (local address 127.13.149.193:41875)
0924 13:51:12.207562 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0924 13:51:12.207575 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0924 13:51:12.207639 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:51:12.208526 (+ 887us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:51:12.208529 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:51:12.208894 (+ 365us) client_negotiation.cc:189] Negotiated authn=SASL
0924 13:51:12.209107 (+ 213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:51:12.209113 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:51:12.210180 (+ 1067us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:51:12.210181 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:51:12.210942 (+ 761us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:51:12.210943 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:51:12.211512 (+ 569us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:51:12.211513 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:51:12.211609 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:51:12.212223 (+ 614us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 13:51:12.212228 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 13:51:12.214046 (+ 1818us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 13:51:12.215911 (+ 1865us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 13:51:12.215912 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 13:51:12.215912 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 13:51:12.216113 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 13:51:12.216394 (+ 281us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 13:51:12.216394 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 13:51:12.216395 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 13:51:12.216466 (+ 71us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 13:51:12.216829 (+ 363us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 13:51:12.216835 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 13:51:12.216985 (+ 150us) client_negotiation.cc:769] Sending connection context
0924 13:51:12.217075 (+ 90us) client_negotiation.cc:240] Negotiation successful
0924 13:51:12.217263 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43,"mutex_wait_us":12}
I20260924 13:51:12.217617 30926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:12.207519 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.194:37117 (local address 127.13.149.193:49551)
0924 13:51:12.207678 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0924 13:51:12.207688 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 13:51:12.207765 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:51:12.208526 (+ 761us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:51:12.208529 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:51:12.208895 (+ 366us) client_negotiation.cc:189] Negotiated authn=SASL
0924 13:51:12.209107 (+ 212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:51:12.209113 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:51:12.210172 (+ 1059us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:51:12.210173 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:51:12.210943 (+ 770us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:51:12.210945 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:51:12.211505 (+ 560us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:51:12.211506 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:51:12.211611 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:51:12.212168 (+ 557us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 13:51:12.212178 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0924 13:51:12.213726 (+ 1548us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 13:51:12.215820 (+ 2094us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 13:51:12.215826 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 13:51:12.215828 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 13:51:12.216111 (+ 283us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 13:51:12.216361 (+ 250us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 13:51:12.216362 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 13:51:12.216362 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 13:51:12.216467 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 13:51:12.216849 (+ 382us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 13:51:12.216850 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 13:51:12.216984 (+ 134us) client_negotiation.cc:769] Sending connection context
0924 13:51:12.217075 (+ 91us) client_negotiation.cc:240] Negotiation successful
0924 13:51:12.217341 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"mutex_wait_us":175,"thread_start_us":62,"threads_started":1}
W20260924 13:51:12.218376 30540 leader_election.cc:341] T e6e9a3195d204d9495b51fa621106833 P efba9ffdd4eb42929192092d77763fcc [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer dff77ad6341d44f8a14ec7f008ba9d73 (127.13.149.194:37117): Not found: Tablet not found: e6e9a3195d204d9495b51fa621106833
W20260924 13:51:12.218492 30538 leader_election.cc:341] T e6e9a3195d204d9495b51fa621106833 P efba9ffdd4eb42929192092d77763fcc [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1c5172f7acb64efd980cb3d3e77d1851 (127.13.149.195:42897): Not found: Tablet not found: e6e9a3195d204d9495b51fa621106833
I20260924 13:51:12.218559 30538 leader_election.cc:302] T e6e9a3195d204d9495b51fa621106833 P efba9ffdd4eb42929192092d77763fcc [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efba9ffdd4eb42929192092d77763fcc; no voters: 1c5172f7acb64efd980cb3d3e77d1851, dff77ad6341d44f8a14ec7f008ba9d73
I20260924 13:51:12.218717 30925 raft_consensus.cc:2726] T e6e9a3195d204d9495b51fa621106833 P efba9ffdd4eb42929192092d77763fcc [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260924 13:51:12.936430 30929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:12.936192 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:38197 (local address 127.13.149.193:50991)
0924 13:51:12.936334 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0924 13:51:12.936379 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.149.254:38197: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":83,"threads_started":1}
I20260924 13:51:12.991047 30930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:12.990833 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:38197 (local address 127.13.149.194:60141)
0924 13:51:12.990967 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0924 13:51:12.991008 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.149.254:38197: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":60,"threads_started":1}
I20260924 13:51:13.121176 30931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:13.120976 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:38197 (local address 127.13.149.195:33799)
0924 13:51:13.121088 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0924 13:51:13.121132 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.149.254:38197: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":94,"thread_start_us":58,"threads_started":1}
I20260924 13:51:13.937222 30932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:13.936987 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:38197 (local address 127.13.149.193:42801)
0924 13:51:13.937122 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0924 13:51:13.937161 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.149.254:38197: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":68,"threads_started":1}
I20260924 13:51:13.991690 30933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:13.991474 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:38197 (local address 127.13.149.194:37391)
0924 13:51:13.991610 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0924 13:51:13.991649 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.149.254:38197: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":63,"threads_started":1}
I20260924 13:51:14.121827 30934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:14.121614 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:38197 (local address 127.13.149.195:35895)
0924 13:51:14.121743 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0924 13:51:14.121785 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.149.254:38197: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":58,"threads_started":1}
I20260924 13:51:14.418341 30935 raft_consensus.cc:482] T e6e9a3195d204d9495b51fa621106833 P efba9ffdd4eb42929192092d77763fcc [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 13:51:14.418449 30935 raft_consensus.cc:504] T e6e9a3195d204d9495b51fa621106833 P efba9ffdd4eb42929192092d77763fcc [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c5172f7acb64efd980cb3d3e77d1851" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 42897 } } peers { permanent_uuid: "dff77ad6341d44f8a14ec7f008ba9d73" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 37117 } } peers { permanent_uuid: "efba9ffdd4eb42929192092d77763fcc" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 35539 } }
I20260924 13:51:14.418627 30935 leader_election.cc:288] T e6e9a3195d204d9495b51fa621106833 P efba9ffdd4eb42929192092d77763fcc [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1c5172f7acb64efd980cb3d3e77d1851 (127.13.149.195:42897), dff77ad6341d44f8a14ec7f008ba9d73 (127.13.149.194:37117)
W20260924 13:51:14.418998 30538 leader_election.cc:341] T e6e9a3195d204d9495b51fa621106833 P efba9ffdd4eb42929192092d77763fcc [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1c5172f7acb64efd980cb3d3e77d1851 (127.13.149.195:42897): Not found: Tablet not found: e6e9a3195d204d9495b51fa621106833
W20260924 13:51:14.419090 30540 leader_election.cc:341] T e6e9a3195d204d9495b51fa621106833 P efba9ffdd4eb42929192092d77763fcc [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer dff77ad6341d44f8a14ec7f008ba9d73 (127.13.149.194:37117): Not found: Tablet not found: e6e9a3195d204d9495b51fa621106833
I20260924 13:51:14.419173 30540 leader_election.cc:302] T e6e9a3195d204d9495b51fa621106833 P efba9ffdd4eb42929192092d77763fcc [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efba9ffdd4eb42929192092d77763fcc; no voters: 1c5172f7acb64efd980cb3d3e77d1851, dff77ad6341d44f8a14ec7f008ba9d73
I20260924 13:51:14.419250 30935 raft_consensus.cc:2726] T e6e9a3195d204d9495b51fa621106833 P efba9ffdd4eb42929192092d77763fcc [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260924 13:51:14.937863 30936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:14.937613 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:38197 (local address 127.13.149.193:45979)
0924 13:51:14.937764 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0924 13:51:14.937809 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.149.254:38197: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":61,"threads_started":1}
I20260924 13:51:14.992354 30937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:14.992158 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:38197 (local address 127.13.149.194:49721)
0924 13:51:14.992270 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0924 13:51:14.992309 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.149.254:38197: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96,"thread_start_us":60,"threads_started":1}
I20260924 13:51:15.122536 30938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:15.122295 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:38197 (local address 127.13.149.195:41389)
0924 13:51:15.122444 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0924 13:51:15.122489 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.149.254:38197: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":72,"threads_started":1}
I20260924 13:51:15.938617 30939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:15.938361 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:38197 (local address 127.13.149.193:40049)
0924 13:51:15.938525 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0924 13:51:15.938567 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.149.254:38197: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":70,"threads_started":1}
I20260924 13:51:15.993005 30940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:15.992774 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:38197 (local address 127.13.149.194:38871)
0924 13:51:15.992899 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0924 13:51:15.992962 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.149.254:38197: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":57,"threads_started":1}
I20260924 13:51:16.123381 30941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:16.123104 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:38197 (local address 127.13.149.195:36037)
0924 13:51:16.123288 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0924 13:51:16.123333 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.149.254:38197: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":85,"threads_started":1}
I20260924 13:51:16.939383 30942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:16.939074 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:38197 (local address 127.13.149.193:45705)
0924 13:51:16.939294 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0924 13:51:16.939334 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.149.254:38197: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":126,"threads_started":1}
I20260924 13:51:16.993613 30943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:16.993409 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:38197 (local address 127.13.149.194:47565)
0924 13:51:16.993537 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0924 13:51:16.993568 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.149.254:38197: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":70,"threads_started":1}
I20260924 13:51:17.124192 30944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:17.123935 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:38197 (local address 127.13.149.195:43413)
0924 13:51:17.124098 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0924 13:51:17.124143 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.149.254:38197: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":76,"threads_started":1}
I20260924 13:51:17.484133 30945 raft_consensus.cc:482] T e6e9a3195d204d9495b51fa621106833 P efba9ffdd4eb42929192092d77763fcc [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 13:51:17.484249 30945 raft_consensus.cc:504] T e6e9a3195d204d9495b51fa621106833 P efba9ffdd4eb42929192092d77763fcc [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c5172f7acb64efd980cb3d3e77d1851" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 42897 } } peers { permanent_uuid: "dff77ad6341d44f8a14ec7f008ba9d73" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 37117 } } peers { permanent_uuid: "efba9ffdd4eb42929192092d77763fcc" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 35539 } }
I20260924 13:51:17.484397 30945 leader_election.cc:288] T e6e9a3195d204d9495b51fa621106833 P efba9ffdd4eb42929192092d77763fcc [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1c5172f7acb64efd980cb3d3e77d1851 (127.13.149.195:42897), dff77ad6341d44f8a14ec7f008ba9d73 (127.13.149.194:37117)
W20260924 13:51:17.484834 30538 leader_election.cc:341] T e6e9a3195d204d9495b51fa621106833 P efba9ffdd4eb42929192092d77763fcc [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1c5172f7acb64efd980cb3d3e77d1851 (127.13.149.195:42897): Not found: Tablet not found: e6e9a3195d204d9495b51fa621106833
W20260924 13:51:17.484938 30540 leader_election.cc:341] T e6e9a3195d204d9495b51fa621106833 P efba9ffdd4eb42929192092d77763fcc [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer dff77ad6341d44f8a14ec7f008ba9d73 (127.13.149.194:37117): Not found: Tablet not found: e6e9a3195d204d9495b51fa621106833
I20260924 13:51:17.485001 30540 leader_election.cc:302] T e6e9a3195d204d9495b51fa621106833 P efba9ffdd4eb42929192092d77763fcc [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efba9ffdd4eb42929192092d77763fcc; no voters: 1c5172f7acb64efd980cb3d3e77d1851, dff77ad6341d44f8a14ec7f008ba9d73
I20260924 13:51:17.485066 30945 raft_consensus.cc:2726] T e6e9a3195d204d9495b51fa621106833 P efba9ffdd4eb42929192092d77763fcc [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260924 13:51:17.940105 30946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:17.939887 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:38197 (local address 127.13.149.193:54479)
0924 13:51:17.940022 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0924 13:51:17.940057 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.149.254:38197: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":74,"threads_started":1}
I20260924 13:51:17.994190 30947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:17.993999 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:38197 (local address 127.13.149.194:38709)
0924 13:51:17.994108 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0924 13:51:17.994144 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.149.254:38197: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93,"thread_start_us":60,"threads_started":1}
I20260924 13:51:18.124994 30948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:18.124716 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:38197 (local address 127.13.149.195:47597)
0924 13:51:18.124866 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0924 13:51:18.124908 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.149.254:38197: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":74,"threads_started":1}
I20260924 13:51:18.940908 30949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:18.940668 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:38197 (local address 127.13.149.193:42477)
0924 13:51:18.940806 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0924 13:51:18.940852 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.149.254:38197: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":81,"threads_started":1}
I20260924 13:51:18.994843 30950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:18.994634 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:38197 (local address 127.13.149.194:47379)
0924 13:51:18.994763 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0924 13:51:18.994798 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.149.254:38197: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":60,"threads_started":1}
I20260924 13:51:19.125779 30951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:19.125514 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:38197 (local address 127.13.149.195:48259)
0924 13:51:19.125684 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0924 13:51:19.125730 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.149.254:38197: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":90,"threads_started":1}
I20260924 13:51:19.941711 30952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:19.941479 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:38197 (local address 127.13.149.193:54165)
0924 13:51:19.941616 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0924 13:51:19.941664 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.149.254:38197: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":75,"threads_started":1}
I20260924 13:51:19.995530 30953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:19.995323 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:38197 (local address 127.13.149.194:37927)
0924 13:51:19.995446 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0924 13:51:19.995483 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.149.254:38197: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":61,"threads_started":1}
I20260924 13:51:20.126637 30954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:20.126319 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:38197 (local address 127.13.149.195:47793)
0924 13:51:20.126525 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0924 13:51:20.126585 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.149.254:38197: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":71,"threads_started":1}
I20260924 13:51:20.786288 30955 raft_consensus.cc:482] T e6e9a3195d204d9495b51fa621106833 P efba9ffdd4eb42929192092d77763fcc [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 13:51:20.786389 30955 raft_consensus.cc:504] T e6e9a3195d204d9495b51fa621106833 P efba9ffdd4eb42929192092d77763fcc [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c5172f7acb64efd980cb3d3e77d1851" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 42897 } } peers { permanent_uuid: "dff77ad6341d44f8a14ec7f008ba9d73" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 37117 } } peers { permanent_uuid: "efba9ffdd4eb42929192092d77763fcc" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 35539 } }
I20260924 13:51:20.786571 30955 leader_election.cc:288] T e6e9a3195d204d9495b51fa621106833 P efba9ffdd4eb42929192092d77763fcc [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1c5172f7acb64efd980cb3d3e77d1851 (127.13.149.195:42897), dff77ad6341d44f8a14ec7f008ba9d73 (127.13.149.194:37117)
W20260924 13:51:20.786934 30538 leader_election.cc:341] T e6e9a3195d204d9495b51fa621106833 P efba9ffdd4eb42929192092d77763fcc [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1c5172f7acb64efd980cb3d3e77d1851 (127.13.149.195:42897): Not found: Tablet not found: e6e9a3195d204d9495b51fa621106833
W20260924 13:51:20.787016 30540 leader_election.cc:341] T e6e9a3195d204d9495b51fa621106833 P efba9ffdd4eb42929192092d77763fcc [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer dff77ad6341d44f8a14ec7f008ba9d73 (127.13.149.194:37117): Not found: Tablet not found: e6e9a3195d204d9495b51fa621106833
I20260924 13:51:20.787076 30540 leader_election.cc:302] T e6e9a3195d204d9495b51fa621106833 P efba9ffdd4eb42929192092d77763fcc [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efba9ffdd4eb42929192092d77763fcc; no voters: 1c5172f7acb64efd980cb3d3e77d1851, dff77ad6341d44f8a14ec7f008ba9d73
I20260924 13:51:20.787138 30955 raft_consensus.cc:2726] T e6e9a3195d204d9495b51fa621106833 P efba9ffdd4eb42929192092d77763fcc [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260924 13:51:20.942538 30956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:20.942313 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:38197 (local address 127.13.149.193:54375)
0924 13:51:20.942439 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0924 13:51:20.942487 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.149.254:38197: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":63,"threads_started":1}
I20260924 13:51:20.996253 30957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:20.996012 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:38197 (local address 127.13.149.194:54641)
0924 13:51:20.996146 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0924 13:51:20.996202 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.149.254:38197: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":73,"threads_started":1}
I20260924 13:51:21.127378 30958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:21.127179 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:38197 (local address 127.13.149.195:58779)
0924 13:51:21.127294 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0924 13:51:21.127330 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.149.254:38197: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":62,"threads_started":1}
I20260924 13:51:21.943214 30959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:21.943010 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:38197 (local address 127.13.149.193:56091)
0924 13:51:21.943127 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0924 13:51:21.943164 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.149.254:38197: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":57,"threads_started":1}
I20260924 13:51:21.997012 30960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:21.996762 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:38197 (local address 127.13.149.194:43023)
0924 13:51:21.996898 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0924 13:51:21.996961 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.149.254:38197: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":56,"threads_started":1}
I20260924 13:51:22.128063 30961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:22.127851 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:38197 (local address 127.13.149.195:49161)
0924 13:51:22.127982 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0924 13:51:22.128007 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.149.254:38197: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":65,"threads_started":1}
I20260924 13:51:22.944041 30962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:22.943790 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:38197 (local address 127.13.149.193:60987)
0924 13:51:22.943945 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0924 13:51:22.943993 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.149.254:38197: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":74,"threads_started":1}
I20260924 13:51:22.997687 30963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:22.997457 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:38197 (local address 127.13.149.194:55231)
0924 13:51:22.997596 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0924 13:51:22.997638 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.149.254:38197: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":62,"threads_started":1}
I20260924 13:51:23.128662 30964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:23.128465 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:38197 (local address 127.13.149.195:38259)
0924 13:51:23.128582 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0924 13:51:23.128617 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.149.254:38197: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":62,"threads_started":1}
I20260924 13:51:23.944767 30965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:23.944523 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:38197 (local address 127.13.149.193:35177)
0924 13:51:23.944680 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0924 13:51:23.944721 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.149.254:38197: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":89,"threads_started":1}
I20260924 13:51:23.998500 30966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:23.998262 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:38197 (local address 127.13.149.194:38205)
0924 13:51:23.998409 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0924 13:51:23.998454 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.149.254:38197: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":75,"threads_started":1}
I20260924 13:51:24.129405 30967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:24.129128 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:38197 (local address 127.13.149.195:58637)
0924 13:51:24.129318 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0924 13:51:24.129360 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.149.254:38197: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":60,"threads_started":1}
I20260924 13:51:24.794411 30968 raft_consensus.cc:482] T e6e9a3195d204d9495b51fa621106833 P efba9ffdd4eb42929192092d77763fcc [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 13:51:24.794508 30968 raft_consensus.cc:504] T e6e9a3195d204d9495b51fa621106833 P efba9ffdd4eb42929192092d77763fcc [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c5172f7acb64efd980cb3d3e77d1851" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 42897 } } peers { permanent_uuid: "dff77ad6341d44f8a14ec7f008ba9d73" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 37117 } } peers { permanent_uuid: "efba9ffdd4eb42929192092d77763fcc" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 35539 } }
I20260924 13:51:24.794697 30968 leader_election.cc:288] T e6e9a3195d204d9495b51fa621106833 P efba9ffdd4eb42929192092d77763fcc [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1c5172f7acb64efd980cb3d3e77d1851 (127.13.149.195:42897), dff77ad6341d44f8a14ec7f008ba9d73 (127.13.149.194:37117)
W20260924 13:51:24.795048 30538 leader_election.cc:341] T e6e9a3195d204d9495b51fa621106833 P efba9ffdd4eb42929192092d77763fcc [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1c5172f7acb64efd980cb3d3e77d1851 (127.13.149.195:42897): Not found: Tablet not found: e6e9a3195d204d9495b51fa621106833
W20260924 13:51:24.795135 30540 leader_election.cc:341] T e6e9a3195d204d9495b51fa621106833 P efba9ffdd4eb42929192092d77763fcc [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer dff77ad6341d44f8a14ec7f008ba9d73 (127.13.149.194:37117): Not found: Tablet not found: e6e9a3195d204d9495b51fa621106833
I20260924 13:51:24.795179 30540 leader_election.cc:302] T e6e9a3195d204d9495b51fa621106833 P efba9ffdd4eb42929192092d77763fcc [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efba9ffdd4eb42929192092d77763fcc; no voters: 1c5172f7acb64efd980cb3d3e77d1851, dff77ad6341d44f8a14ec7f008ba9d73
I20260924 13:51:24.795248 30968 raft_consensus.cc:2726] T e6e9a3195d204d9495b51fa621106833 P efba9ffdd4eb42929192092d77763fcc [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260924 13:51:24.945468 30969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:24.945233 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:38197 (local address 127.13.149.193:57181)
0924 13:51:24.945381 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0924 13:51:24.945421 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.149.254:38197: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":60,"threads_started":1}
I20260924 13:51:24.999153 30970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:24.998955 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:38197 (local address 127.13.149.194:43835)
0924 13:51:24.999075 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0924 13:51:24.999108 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.149.254:38197: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":57,"threads_started":1}
I20260924 13:51:25.130270 30971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:25.130039 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:38197 (local address 127.13.149.195:41817)
0924 13:51:25.130180 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0924 13:51:25.130224 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.149.254:38197: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":56,"threads_started":1}
I20260924 13:51:25.946329 30972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:25.946084 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:38197 (local address 127.13.149.193:37365)
0924 13:51:25.946235 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0924 13:51:25.946282 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.149.254:38197: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":63,"threads_started":1}
I20260924 13:51:25.999950 30973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:25.999712 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:38197 (local address 127.13.149.194:34183)
0924 13:51:25.999858 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0924 13:51:25.999903 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.149.254:38197: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":75,"threads_started":1}
I20260924 13:51:26.127144 13911 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJN9WRF/build/release/bin/kudu with pid 30488
I20260924 13:51:26.131045 30974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:26.130838 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:38197 (local address 127.13.149.195:46795)
0924 13:51:26.130961 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0924 13:51:26.131001 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.149.254:38197: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":63,"threads_started":1}
I20260924 13:51:26.131940 13911 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJN9WRF/build/release/bin/kudu with pid 30656
I20260924 13:51:26.135914 13911 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJN9WRF/build/release/bin/kudu with pid 30789
2026-09-24T13:51:26Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (58361 ms)
[----------] 30 tests from SecurityITest (257101 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790257629041645-13911-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 13:51:26 dist-test-slave-sjl2 krb5kdc[30977](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 13:51:26 dist-test-slave-sjl2 krb5kdc[30977](info): set up 2 sockets
Sep 24 13:51:26 dist-test-slave-sjl2 krb5kdc[30977](info): commencing operation
krb5kdc: starting...
W20260924 13:51:27.089653 13911 mini_kdc.cc:121] Time spent starting KDC: real 0.935s user 0.002s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790257629041645-13911-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790257629041645-13911-0/krb5kdc/test-user.keytab.
Sep 24 13:51:27 dist-test-slave-sjl2 krb5kdc[30977](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257887, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T13:51:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T13:51:27Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.149.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.149.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.149.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.254.keytab.
Entry for principal kudu/127.13.149.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.254.keytab.
I20260924 13:51:27.205747 13911 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJN9WRF/build/release/bin/kudu
/tmp/dist-test-taskJN9WRF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790257629041645-13911-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790257629041645-13911-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790257629041645-13911-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790257629041645-13911-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.149.254:38609
--webserver_interface=127.13.149.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.254.keytab
--principal=kudu/127.13.149.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.149.212:45807
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.149.254:38609
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790257629041645-13911-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790257629041645-13911-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790257629041645-13911-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 13:51:27.286655 30990 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 13:51:27.286876 30990 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 13:51:27.286899 30990 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 13:51:27.288051 30990 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 13:51:27.288098 30990 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 13:51:27.288111 30990 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 13:51:27.288122 30990 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 13:51:27.288133 30990 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 13:51:27.289698 30990 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.149.212:45807
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790257629041645-13911-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790257629041645-13911-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.149.254:38609
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.254.keytab
--principal=kudu/127.13.149.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.149.254:38609
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790257629041645-13911-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.149.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30990
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790257629041645-13911-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 12:29:37 UTC on ccycloud.quasar-iqzbxn.root.comops.site
build id 4670106
I20260924 13:51:27.289937 30990 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 13:51:27.290120 30990 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260924 13:51:27.295085 30990 server_base.cc:1034] running on GCE node
W20260924 13:51:27.295288 30998 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 13:51:27.295094 30995 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 13:51:27.295410 30996 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 13:51:27.295809 30990 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 13:51:27.296152 30990 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 13:51:27.297305 30990 hybrid_clock.cc:648] HybridClock initialized: now 1790257887297268 us; error 54 us; skew 500 ppm
Sep 24 13:51:27 dist-test-slave-sjl2 krb5kdc[30977](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257887, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.149.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 13:51:27.310637 30990 init.cc:377] Logged in from keytab as kudu/127.13.149.254@KRBTEST.COM (short username kudu)
I20260924 13:51:27.311110 30990 webserver.cc:460] Webserver started at http://127.13.149.254:33387/ using document root <none> and password file <none>
I20260924 13:51:27.311317 30990 fs_manager.cc:359] Metadata directory not provided
I20260924 13:51:27.311362 30990 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 13:51:27.311461 30990 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 13:51:27.312537 30990 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790257629041645-13911-0/minicluster-data/master-0/data/instance:
uuid: "3917537cb6e24ee18ec0992c6830480c"
format_stamp: "Formatted at 2026-09-24 13:51:27 on dist-test-slave-sjl2"
server_key: "ab9a126c98d156a6314297f29c393086"
server_key_iv: "e6f9f5d0fcae046fec359dc32cbc44cb"
server_key_version: "encryptionkey@0"
I20260924 13:51:27.312887 30990 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790257629041645-13911-0/minicluster-data/master-0/wal/instance:
uuid: "3917537cb6e24ee18ec0992c6830480c"
format_stamp: "Formatted at 2026-09-24 13:51:27 on dist-test-slave-sjl2"
server_key: "ab9a126c98d156a6314297f29c393086"
server_key_iv: "e6f9f5d0fcae046fec359dc32cbc44cb"
server_key_version: "encryptionkey@0"
I20260924 13:51:27.314321 30990 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260924 13:51:27.315127 31005 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 13:51:27.315340 30990 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260924 13:51:27.315435 30990 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790257629041645-13911-0/minicluster-data/master-0/data,/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790257629041645-13911-0/minicluster-data/master-0/wal
uuid: "3917537cb6e24ee18ec0992c6830480c"
format_stamp: "Formatted at 2026-09-24 13:51:27 on dist-test-slave-sjl2"
server_key: "ab9a126c98d156a6314297f29c393086"
server_key_iv: "e6f9f5d0fcae046fec359dc32cbc44cb"
server_key_version: "encryptionkey@0"
I20260924 13:51:27.315519 30990 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790257629041645-13911-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790257629041645-13911-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790257629041645-13911-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 13:51:27.325570 30990 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 13:51:27.327788 30990 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 13:51:27.327913 30990 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 13:51:27.331941 30990 rpc_server.cc:307] RPC server started. Bound to: 127.13.149.254:38609
I20260924 13:51:27.331976 31057 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.149.254:38609 every 8 connection(s)
I20260924 13:51:27.332337 30990 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790257629041645-13911-0/minicluster-data/master-0/data/info.pb
I20260924 13:51:27.333245 31058 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 13:51:27.335532 31058 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3917537cb6e24ee18ec0992c6830480c: Bootstrap starting.
I20260924 13:51:27.336069 31058 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3917537cb6e24ee18ec0992c6830480c: Neither blocks nor log segments found. Creating new log.
I20260924 13:51:27.336310 31058 log.cc:824] T 00000000000000000000000000000000 P 3917537cb6e24ee18ec0992c6830480c: Log is configured to *not* fsync() on all Append() calls
I20260924 13:51:27.337085 31058 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3917537cb6e24ee18ec0992c6830480c: No bootstrap required, opened a new log
I20260924 13:51:27.338639 31058 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3917537cb6e24ee18ec0992c6830480c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3917537cb6e24ee18ec0992c6830480c" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 38609 } }
I20260924 13:51:27.338779 31058 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3917537cb6e24ee18ec0992c6830480c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 13:51:27.338810 31058 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3917537cb6e24ee18ec0992c6830480c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3917537cb6e24ee18ec0992c6830480c, State: Initialized, Role: FOLLOWER
I20260924 13:51:27.338923 31058 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3917537cb6e24ee18ec0992c6830480c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3917537cb6e24ee18ec0992c6830480c" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 38609 } }
I20260924 13:51:27.338999 31058 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3917537cb6e24ee18ec0992c6830480c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 13:51:27.339027 31058 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3917537cb6e24ee18ec0992c6830480c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 13:51:27.339061 31058 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3917537cb6e24ee18ec0992c6830480c [term 0 FOLLOWER]: Advancing to term 1
I20260924 13:51:27.339552 31058 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3917537cb6e24ee18ec0992c6830480c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3917537cb6e24ee18ec0992c6830480c" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 38609 } }
I20260924 13:51:27.339638 31058 leader_election.cc:302] T 00000000000000000000000000000000 P 3917537cb6e24ee18ec0992c6830480c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3917537cb6e24ee18ec0992c6830480c; no voters:
I20260924 13:51:27.339768 31058 leader_election.cc:288] T 00000000000000000000000000000000 P 3917537cb6e24ee18ec0992c6830480c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 13:51:27.339831 31061 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3917537cb6e24ee18ec0992c6830480c [term 1 FOLLOWER]: Leader election won for term 1
I20260924 13:51:27.340005 31061 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3917537cb6e24ee18ec0992c6830480c [term 1 LEADER]: Becoming Leader. State: Replica: 3917537cb6e24ee18ec0992c6830480c, State: Running, Role: LEADER
I20260924 13:51:27.340029 31058 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3917537cb6e24ee18ec0992c6830480c [sys.catalog]: configured and running, proceeding with master startup.
I20260924 13:51:27.340098 31061 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3917537cb6e24ee18ec0992c6830480c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3917537cb6e24ee18ec0992c6830480c" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 38609 } }
I20260924 13:51:27.340610 31063 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3917537cb6e24ee18ec0992c6830480c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3917537cb6e24ee18ec0992c6830480c. Latest consensus state: current_term: 1 leader_uuid: "3917537cb6e24ee18ec0992c6830480c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3917537cb6e24ee18ec0992c6830480c" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 38609 } } }
I20260924 13:51:27.340655 13911 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJN9WRF/build/release/bin/kudu as pid 30990
I20260924 13:51:27.340691 31063 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3917537cb6e24ee18ec0992c6830480c [sys.catalog]: This master's current role is: LEADER
I20260924 13:51:27.340754 13911 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790257629041645-13911-0/minicluster-data/master-0/wal/instance
I20260924 13:51:27.340996 13911 external_mini_cluster.cc:1428] Setting key 81b03846b2fb7c8c1b68bdd8b6131aac
I20260924 13:51:27.341202 31062 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3917537cb6e24ee18ec0992c6830480c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3917537cb6e24ee18ec0992c6830480c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3917537cb6e24ee18ec0992c6830480c" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 38609 } } }
I20260924 13:51:27.341290 31062 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3917537cb6e24ee18ec0992c6830480c [sys.catalog]: This master's current role is: LEADER
W20260924 13:51:27.341840 31077 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 3917537cb6e24ee18ec0992c6830480c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260924 13:51:27.341888 31077 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260924 13:51:27.341959 31070 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 13:51:27.342149 31070 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 13:51:27.343690 31070 catalog_manager.cc:1347] Generated new cluster ID: b5312e1fd93f474388f30e5c8aafba41
I20260924 13:51:27.343748 31070 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 24 13:51:27 dist-test-slave-sjl2 krb5kdc[30977](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257887, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.149.254@KRBTEST.COM
I20260924 13:51:27.350570 31070 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 13:51:27.351159 31079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:27.341542 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39690 (local address 127.13.149.254:38609)
0924 13:51:27.341706 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0924 13:51:27.341714 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0924 13:51:27.341740 (+ 26us) server_negotiation.cc:407] Connection header received
0924 13:51:27.341849 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:51:27.341856 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:51:27.341900 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:51:27.342043 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:51:27.342665 (+ 622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:51:27.343878 (+ 1213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:51:27.344657 (+ 779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:51:27.345134 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:51:27.345231 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:51:27.347394 (+ 2163us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:51:27.347399 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:51:27.347399 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 13:51:27.347411 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:51:27.349559 (+ 2148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:51:27.349876 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:51:27.349879 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:51:27.349880 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:51:27.349976 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:51:27.350164 (+ 188us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:51:27.350165 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:51:27.350166 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:51:27.350428 (+ 262us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:51:27.350590 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0924 13:51:27.350821 (+ 231us) server_negotiation.cc:299] Negotiation successful
0924 13:51:27.350964 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123,"thread_start_us":76,"threads_started":1}
I20260924 13:51:27.351261 31070 catalog_manager.cc:1504] Loading token signing keys...
I20260924 13:51:27.368142 31070 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3917537cb6e24ee18ec0992c6830480c: Generated new TSK 0
I20260924 13:51:27.368355 31070 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.13.149.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.149.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.149.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.193.keytab.
Entry for principal kudu/127.13.149.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.193.keytab.
I20260924 13:51:27.399981 13911 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJN9WRF/build/release/bin/kudu
/tmp/dist-test-taskJN9WRF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790257629041645-13911-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790257629041645-13911-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790257629041645-13911-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790257629041645-13911-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.149.193:0
--local_ip_for_outbound_sockets=127.13.149.193
--webserver_interface=127.13.149.193
--webserver_port=0
--tserver_master_addrs=127.13.149.254:38609
--keytab_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.193.keytab
--principal=kudu/127.13.149.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.149.212:45807
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790257629041645-13911-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790257629041645-13911-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790257629041645-13911-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 13:51:27.484162 31084 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 13:51:27.484370 31084 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 13:51:27.484395 31084 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 13:51:27.485637 31084 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 13:51:27.485685 31084 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 13:51:27.485713 31084 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.149.193
I20260924 13:51:27.487246 31084 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.149.212:45807
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790257629041645-13911-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790257629041645-13911-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.193.keytab
--principal=kudu/127.13.149.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.149.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790257629041645-13911-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.149.193
--webserver_port=0
--tserver_master_addrs=127.13.149.254:38609
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31084
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.149.193
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790257629041645-13911-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 12:29:37 UTC on ccycloud.quasar-iqzbxn.root.comops.site
build id 4670106
I20260924 13:51:27.487481 31084 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 13:51:27.487694 31084 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 13:51:27.492648 31092 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 13:51:27.492684 31090 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 13:51:27.492648 31089 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 13:51:27.492861 31084 server_base.cc:1034] running on GCE node
I20260924 13:51:27.493079 31084 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 13:51:27.493335 31084 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 13:51:27.494477 31084 hybrid_clock.cc:648] HybridClock initialized: now 1790257887494462 us; error 29 us; skew 500 ppm
Sep 24 13:51:27 dist-test-slave-sjl2 krb5kdc[30977](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257887, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.149.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 13:51:27.496888 31084 init.cc:377] Logged in from keytab as kudu/127.13.149.193@KRBTEST.COM (short username kudu)
I20260924 13:51:27.497375 31084 webserver.cc:460] Webserver started at http://127.13.149.193:37365/ using document root <none> and password file <none>
I20260924 13:51:27.497568 31084 fs_manager.cc:359] Metadata directory not provided
I20260924 13:51:27.497622 31084 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 13:51:27.497745 31084 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 13:51:27.498765 31084 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790257629041645-13911-0/minicluster-data/ts-0/data/instance:
uuid: "27e4fdd555804c8dbeb3d8ef892cf590"
format_stamp: "Formatted at 2026-09-24 13:51:27 on dist-test-slave-sjl2"
server_key: "044093dd27a28987bc414f62b444306a"
server_key_iv: "ca0cacfc3052e5f7d7035545518de0d4"
server_key_version: "encryptionkey@0"
I20260924 13:51:27.499085 31084 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790257629041645-13911-0/minicluster-data/ts-0/wal/instance:
uuid: "27e4fdd555804c8dbeb3d8ef892cf590"
format_stamp: "Formatted at 2026-09-24 13:51:27 on dist-test-slave-sjl2"
server_key: "044093dd27a28987bc414f62b444306a"
server_key_iv: "ca0cacfc3052e5f7d7035545518de0d4"
server_key_version: "encryptionkey@0"
I20260924 13:51:27.500268 31084 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260924 13:51:27.500907 31099 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 13:51:27.501068 31084 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260924 13:51:27.501152 31084 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790257629041645-13911-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790257629041645-13911-0/minicluster-data/ts-0/wal
uuid: "27e4fdd555804c8dbeb3d8ef892cf590"
format_stamp: "Formatted at 2026-09-24 13:51:27 on dist-test-slave-sjl2"
server_key: "044093dd27a28987bc414f62b444306a"
server_key_iv: "ca0cacfc3052e5f7d7035545518de0d4"
server_key_version: "encryptionkey@0"
I20260924 13:51:27.501222 31084 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790257629041645-13911-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790257629041645-13911-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790257629041645-13911-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 13:51:27.514096 31084 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 13:51:27.516170 31084 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 13:51:27.516307 31084 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 13:51:27.516518 31084 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 13:51:27.516820 31084 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 13:51:27.516853 31084 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 13:51:27.516882 31084 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 13:51:27.516904 31084 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 13:51:27.522543 31084 rpc_server.cc:307] RPC server started. Bound to: 127.13.149.193:39821
I20260924 13:51:27.522601 31212 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.149.193:39821 every 8 connection(s)
I20260924 13:51:27.522923 31084 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790257629041645-13911-0/minicluster-data/ts-0/data/info.pb
I20260924 13:51:27.525990 13911 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJN9WRF/build/release/bin/kudu as pid 31084
I20260924 13:51:27.526098 13911 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790257629041645-13911-0/minicluster-data/ts-0/wal/instance
I20260924 13:51:27.526286 13911 external_mini_cluster.cc:1428] Setting key 2e6ab9f70d88a3ad966b65489e6e1a40
Sep 24 13:51:27 dist-test-slave-sjl2 krb5kdc[30977](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257887, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.149.193@KRBTEST.COM for kudu/127.13.149.254@KRBTEST.COM
I20260924 13:51:27.532286 31079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:27.523601 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.149.193:60665 (local address 127.13.149.254:38609)
0924 13:51:27.523715 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0924 13:51:27.523717 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:51:27.523869 (+ 152us) server_negotiation.cc:407] Connection header received
0924 13:51:27.524364 (+ 495us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:51:27.524365 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:51:27.524378 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:51:27.524432 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:51:27.525383 (+ 951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:51:27.525884 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:51:27.527150 (+ 1266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:51:27.527359 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:51:27.527412 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:51:27.529740 (+ 2328us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:51:27.529741 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:51:27.529742 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 13:51:27.529749 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:51:27.531067 (+ 1318us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:51:27.531424 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:51:27.531425 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:51:27.531426 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:51:27.531478 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:51:27.531690 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:51:27.531691 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:51:27.531691 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:51:27.531803 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:51:27.531859 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0924 13:51:27.532110 (+ 251us) server_negotiation.cc:299] Negotiation successful
0924 13:51:27.532197 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260924 13:51:27.532462 31215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:27.523565 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:38609 (local address 127.13.149.193:60665)
0924 13:51:27.523731 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0924 13:51:27.523773 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0924 13:51:27.524269 (+ 496us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:51:27.524491 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:51:27.524495 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:51:27.525095 (+ 600us) client_negotiation.cc:189] Negotiated authn=SASL
0924 13:51:27.525290 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:51:27.525295 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:51:27.525977 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:51:27.525978 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:51:27.527073 (+ 1095us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:51:27.527076 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:51:27.527426 (+ 350us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:51:27.527427 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:51:27.527550 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:51:27.528033 (+ 483us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 13:51:27.528043 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0924 13:51:27.529654 (+ 1611us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 13:51:27.531142 (+ 1488us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 13:51:27.531148 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 13:51:27.531151 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0924 13:51:27.531361 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 13:51:27.531530 (+ 169us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 13:51:27.531531 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 13:51:27.531531 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 13:51:27.531637 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 13:51:27.531856 (+ 219us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 13:51:27.531862 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 13:51:27.531994 (+ 132us) client_negotiation.cc:769] Sending connection context
0924 13:51:27.532082 (+ 88us) client_negotiation.cc:240] Negotiation successful
0924 13:51:27.532249 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":93,"threads_started":1}
I20260924 13:51:27.532934 31213 heartbeater.cc:347] Connected to a master server at 127.13.149.254:38609
I20260924 13:51:27.533028 31213 heartbeater.cc:464] Registering TS with master...
I20260924 13:51:27.533203 31213 heartbeater.cc:511] Master 127.13.149.254:38609 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.13.149.194@KRBTEST.COM; defaulting to no policy
I20260924 13:51:27.533727 31022 ts_manager.cc:194] Registered new tserver with Master: 27e4fdd555804c8dbeb3d8ef892cf590 (127.13.149.193:39821)
I20260924 13:51:27.534358 31022 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.149.193@KRBTEST.COM'} at 127.13.149.193:60665
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.149.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.149.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.194.keytab.
Entry for principal kudu/127.13.149.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.194.keytab.
I20260924 13:51:27.549892 13911 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJN9WRF/build/release/bin/kudu
/tmp/dist-test-taskJN9WRF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790257629041645-13911-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790257629041645-13911-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790257629041645-13911-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790257629041645-13911-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.149.194:0
--local_ip_for_outbound_sockets=127.13.149.194
--webserver_interface=127.13.149.194
--webserver_port=0
--tserver_master_addrs=127.13.149.254:38609
--keytab_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.194.keytab
--principal=kudu/127.13.149.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.149.212:45807
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790257629041645-13911-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790257629041645-13911-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790257629041645-13911-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 13:51:27.630529 31218 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 13:51:27.630753 31218 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 13:51:27.630786 31218 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 13:51:27.631956 31218 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 13:51:27.632000 31218 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 13:51:27.632035 31218 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.149.194
I20260924 13:51:27.633497 31218 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.149.212:45807
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790257629041645-13911-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790257629041645-13911-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.194.keytab
--principal=kudu/127.13.149.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.149.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790257629041645-13911-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.149.194
--webserver_port=0
--tserver_master_addrs=127.13.149.254:38609
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31218
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.149.194
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790257629041645-13911-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 12:29:37 UTC on ccycloud.quasar-iqzbxn.root.comops.site
build id 4670106
I20260924 13:51:27.633741 31218 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 13:51:27.633965 31218 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 13:51:27.638837 31224 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 13:51:27.638900 31218 server_base.cc:1034] running on GCE node
W20260924 13:51:27.638906 31223 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 13:51:27.638825 31226 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 13:51:27.639216 31218 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 13:51:27.639439 31218 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 13:51:27.640606 31218 hybrid_clock.cc:648] HybridClock initialized: now 1790257887640593 us; error 31 us; skew 500 ppm
Sep 24 13:51:27 dist-test-slave-sjl2 krb5kdc[30977](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257887, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.149.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 13:51:27.643014 31218 init.cc:377] Logged in from keytab as kudu/127.13.149.194@KRBTEST.COM (short username kudu)
I20260924 13:51:27.643455 31218 webserver.cc:460] Webserver started at http://127.13.149.194:36131/ using document root <none> and password file <none>
I20260924 13:51:27.643651 31218 fs_manager.cc:359] Metadata directory not provided
I20260924 13:51:27.643693 31218 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 13:51:27.643816 31218 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 13:51:27.644817 31218 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790257629041645-13911-0/minicluster-data/ts-1/data/instance:
uuid: "446c38483d164f7784169a299e102035"
format_stamp: "Formatted at 2026-09-24 13:51:27 on dist-test-slave-sjl2"
server_key: "d5f148e565671513faa0b0a8242b4668"
server_key_iv: "784264f40c31cb2ff290253c7817c760"
server_key_version: "encryptionkey@0"
I20260924 13:51:27.645195 31218 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790257629041645-13911-0/minicluster-data/ts-1/wal/instance:
uuid: "446c38483d164f7784169a299e102035"
format_stamp: "Formatted at 2026-09-24 13:51:27 on dist-test-slave-sjl2"
server_key: "d5f148e565671513faa0b0a8242b4668"
server_key_iv: "784264f40c31cb2ff290253c7817c760"
server_key_version: "encryptionkey@0"
I20260924 13:51:27.646407 31218 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260924 13:51:27.647040 31233 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 13:51:27.647176 31218 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260924 13:51:27.647251 31218 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790257629041645-13911-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790257629041645-13911-0/minicluster-data/ts-1/wal
uuid: "446c38483d164f7784169a299e102035"
format_stamp: "Formatted at 2026-09-24 13:51:27 on dist-test-slave-sjl2"
server_key: "d5f148e565671513faa0b0a8242b4668"
server_key_iv: "784264f40c31cb2ff290253c7817c760"
server_key_version: "encryptionkey@0"
I20260924 13:51:27.647324 31218 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790257629041645-13911-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790257629041645-13911-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790257629041645-13911-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 13:51:27.664647 31218 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 13:51:27.666465 31218 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 13:51:27.666594 31218 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 13:51:27.666814 31218 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 13:51:27.667102 31218 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 13:51:27.667135 31218 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 13:51:27.667158 31218 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 13:51:27.667181 31218 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 13:51:27.672771 31218 rpc_server.cc:307] RPC server started. Bound to: 127.13.149.194:35009
I20260924 13:51:27.672827 31346 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.149.194:35009 every 8 connection(s)
I20260924 13:51:27.673122 31218 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790257629041645-13911-0/minicluster-data/ts-1/data/info.pb
I20260924 13:51:27.674520 13911 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJN9WRF/build/release/bin/kudu as pid 31218
I20260924 13:51:27.674638 13911 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790257629041645-13911-0/minicluster-data/ts-1/wal/instance
I20260924 13:51:27.674841 13911 external_mini_cluster.cc:1428] Setting key ffdb62cf4f4d3f39d08a9a820e016c42
Sep 24 13:51:27 dist-test-slave-sjl2 krb5kdc[30977](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257887, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.149.194@KRBTEST.COM for kudu/127.13.149.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.13.149.195@KRBTEST.COM; defaulting to no policy
I20260924 13:51:27.682227 31079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:27.673768 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.149.194:52361 (local address 127.13.149.254:38609)
0924 13:51:27.673827 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0924 13:51:27.673830 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 13:51:27.674039 (+ 209us) server_negotiation.cc:407] Connection header received
0924 13:51:27.674517 (+ 478us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:51:27.674519 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:51:27.674532 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:51:27.674596 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:51:27.675557 (+ 961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:51:27.676237 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:51:27.677267 (+ 1030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:51:27.677471 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:51:27.677532 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:51:27.679546 (+ 2014us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:51:27.679547 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:51:27.679547 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 13:51:27.679555 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:51:27.680865 (+ 1310us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:51:27.681349 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:51:27.681350 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:51:27.681350 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:51:27.681406 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:51:27.681621 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:51:27.681622 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:51:27.681622 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:51:27.681739 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:51:27.681794 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0924 13:51:27.682048 (+ 254us) server_negotiation.cc:299] Negotiation successful
0924 13:51:27.682134 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260924 13:51:27.682411 31349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:27.673759 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:38609 (local address 127.13.149.194:52361)
0924 13:51:27.673930 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0924 13:51:27.673965 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0924 13:51:27.674433 (+ 468us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:51:27.674697 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:51:27.674702 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:51:27.675271 (+ 569us) client_negotiation.cc:189] Negotiated authn=SASL
0924 13:51:27.675476 (+ 205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:51:27.675480 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:51:27.676352 (+ 872us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:51:27.676353 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:51:27.677183 (+ 830us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:51:27.677186 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:51:27.677546 (+ 360us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:51:27.677547 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:51:27.677663 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:51:27.678144 (+ 481us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 13:51:27.678151 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0924 13:51:27.679456 (+ 1305us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 13:51:27.680989 (+ 1533us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 13:51:27.680997 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 13:51:27.681000 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0924 13:51:27.681276 (+ 276us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 13:51:27.681461 (+ 185us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 13:51:27.681462 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 13:51:27.681462 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 13:51:27.681565 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 13:51:27.681792 (+ 227us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 13:51:27.681798 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 13:51:27.681935 (+ 137us) client_negotiation.cc:769] Sending connection context
0924 13:51:27.682021 (+ 86us) client_negotiation.cc:240] Negotiation successful
0924 13:51:27.682188 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":107,"threads_started":1}
I20260924 13:51:27.682858 31347 heartbeater.cc:347] Connected to a master server at 127.13.149.254:38609
I20260924 13:51:27.682950 31347 heartbeater.cc:464] Registering TS with master...
I20260924 13:51:27.683144 31347 heartbeater.cc:511] Master 127.13.149.254:38609 requested a full tablet report, sending...
I20260924 13:51:27.683481 31022 ts_manager.cc:194] Registered new tserver with Master: 446c38483d164f7784169a299e102035 (127.13.149.194:35009)
I20260924 13:51:27.683836 31022 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.149.194@KRBTEST.COM'} at 127.13.149.194:52361
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.149.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.149.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.195.keytab.
Entry for principal kudu/127.13.149.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.195.keytab.
I20260924 13:51:27.700261 13911 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJN9WRF/build/release/bin/kudu
/tmp/dist-test-taskJN9WRF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790257629041645-13911-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790257629041645-13911-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790257629041645-13911-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790257629041645-13911-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.149.195:0
--local_ip_for_outbound_sockets=127.13.149.195
--webserver_interface=127.13.149.195
--webserver_port=0
--tserver_master_addrs=127.13.149.254:38609
--keytab_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.195.keytab
--principal=kudu/127.13.149.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.149.212:45807
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790257629041645-13911-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790257629041645-13911-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790257629041645-13911-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 13:51:27.781574 31352 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 13:51:27.781800 31352 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 13:51:27.781826 31352 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 13:51:27.783058 31352 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 13:51:27.783113 31352 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 13:51:27.783140 31352 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.149.195
I20260924 13:51:27.784694 31352 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.149.212:45807
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790257629041645-13911-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790257629041645-13911-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.195.keytab
--principal=kudu/127.13.149.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.149.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790257629041645-13911-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.149.195
--webserver_port=0
--tserver_master_addrs=127.13.149.254:38609
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31352
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.149.195
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790257629041645-13911-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 12:29:37 UTC on ccycloud.quasar-iqzbxn.root.comops.site
build id 4670106
I20260924 13:51:27.784981 31352 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 13:51:27.785220 31352 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 13:51:27.790196 31360 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 13:51:27.790354 31352 server_base.cc:1034] running on GCE node
W20260924 13:51:27.790226 31358 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 13:51:27.790215 31357 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 13:51:27.790661 31352 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 13:51:27.790928 31352 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 13:51:27.792068 31352 hybrid_clock.cc:648] HybridClock initialized: now 1790257887792056 us; error 30 us; skew 500 ppm
Sep 24 13:51:27 dist-test-slave-sjl2 krb5kdc[30977](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257887, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.149.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 13:51:27.794682 31352 init.cc:377] Logged in from keytab as kudu/127.13.149.195@KRBTEST.COM (short username kudu)
I20260924 13:51:27.795127 31352 webserver.cc:460] Webserver started at http://127.13.149.195:46437/ using document root <none> and password file <none>
I20260924 13:51:27.795364 31352 fs_manager.cc:359] Metadata directory not provided
I20260924 13:51:27.795411 31352 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 13:51:27.795521 31352 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 13:51:27.796648 31352 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790257629041645-13911-0/minicluster-data/ts-2/data/instance:
uuid: "4074d292e57d49cabc2e8ccc6a26a4cd"
format_stamp: "Formatted at 2026-09-24 13:51:27 on dist-test-slave-sjl2"
server_key: "7683033448753af38ce80bb284e56f37"
server_key_iv: "26fb27158d1b24adecfb3f22b3f3badd"
server_key_version: "encryptionkey@0"
I20260924 13:51:27.797005 31352 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790257629041645-13911-0/minicluster-data/ts-2/wal/instance:
uuid: "4074d292e57d49cabc2e8ccc6a26a4cd"
format_stamp: "Formatted at 2026-09-24 13:51:27 on dist-test-slave-sjl2"
server_key: "7683033448753af38ce80bb284e56f37"
server_key_iv: "26fb27158d1b24adecfb3f22b3f3badd"
server_key_version: "encryptionkey@0"
I20260924 13:51:27.798336 31352 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260924 13:51:27.799082 31367 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 13:51:27.799216 31352 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 13:51:27.799316 31352 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790257629041645-13911-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790257629041645-13911-0/minicluster-data/ts-2/wal
uuid: "4074d292e57d49cabc2e8ccc6a26a4cd"
format_stamp: "Formatted at 2026-09-24 13:51:27 on dist-test-slave-sjl2"
server_key: "7683033448753af38ce80bb284e56f37"
server_key_iv: "26fb27158d1b24adecfb3f22b3f3badd"
server_key_version: "encryptionkey@0"
I20260924 13:51:27.799378 31352 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790257629041645-13911-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790257629041645-13911-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790257629041645-13911-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 13:51:27.812220 31352 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 13:51:27.814320 31352 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 13:51:27.814456 31352 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 13:51:27.814678 31352 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 13:51:27.814965 31352 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 13:51:27.815014 31352 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 13:51:27.815044 31352 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 13:51:27.815068 31352 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 13:51:27.820622 31352 rpc_server.cc:307] RPC server started. Bound to: 127.13.149.195:39491
I20260924 13:51:27.820695 31480 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.149.195:39491 every 8 connection(s)
I20260924 13:51:27.821002 31352 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790257629041645-13911-0/minicluster-data/ts-2/data/info.pb
I20260924 13:51:27.825098 13911 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJN9WRF/build/release/bin/kudu as pid 31352
I20260924 13:51:27.825205 13911 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790257629041645-13911-0/minicluster-data/ts-2/wal/instance
I20260924 13:51:27.825414 13911 external_mini_cluster.cc:1428] Setting key 5ca9291e625f10d9a6c22198aecf451d
Sep 24 13:51:27 dist-test-slave-sjl2 krb5kdc[30977](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257887, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.149.195@KRBTEST.COM for kudu/127.13.149.254@KRBTEST.COM
I20260924 13:51:27.829880 31079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:27.821698 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.149.195:41007 (local address 127.13.149.254:38609)
0924 13:51:27.821745 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0924 13:51:27.821747 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:51:27.821950 (+ 203us) server_negotiation.cc:407] Connection header received
0924 13:51:27.822474 (+ 524us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:51:27.822476 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:51:27.822490 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:51:27.822542 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:51:27.823507 (+ 965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:51:27.824003 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:51:27.825016 (+ 1013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:51:27.825230 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:51:27.825293 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:51:27.827483 (+ 2190us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:51:27.827484 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:51:27.827485 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 13:51:27.827491 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:51:27.828558 (+ 1067us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:51:27.828999 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:51:27.829001 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:51:27.829001 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:51:27.829061 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:51:27.829274 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:51:27.829275 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:51:27.829276 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:51:27.829398 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:51:27.829454 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0924 13:51:27.829709 (+ 255us) server_negotiation.cc:299] Negotiation successful
0924 13:51:27.829791 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260924 13:51:27.830053 31483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:27.821678 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:38609 (local address 127.13.149.195:41007)
0924 13:51:27.821824 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0924 13:51:27.821860 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0924 13:51:27.822361 (+ 501us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:51:27.822610 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:51:27.822614 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:51:27.823203 (+ 589us) client_negotiation.cc:189] Negotiated authn=SASL
0924 13:51:27.823400 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:51:27.823405 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:51:27.824088 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:51:27.824089 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:51:27.824944 (+ 855us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:51:27.824947 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:51:27.825308 (+ 361us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:51:27.825310 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:51:27.825474 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:51:27.826027 (+ 553us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 13:51:27.826038 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0924 13:51:27.827405 (+ 1367us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 13:51:27.828623 (+ 1218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 13:51:27.828629 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 13:51:27.828631 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 13:51:27.828901 (+ 270us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 13:51:27.829113 (+ 212us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 13:51:27.829114 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 13:51:27.829114 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 13:51:27.829221 (+ 107us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 13:51:27.829452 (+ 231us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 13:51:27.829459 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 13:51:27.829593 (+ 134us) client_negotiation.cc:769] Sending connection context
0924 13:51:27.829680 (+ 87us) client_negotiation.cc:240] Negotiation successful
0924 13:51:27.829845 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":92,"thread_start_us":60,"threads_started":1}
I20260924 13:51:27.830432 31481 heartbeater.cc:347] Connected to a master server at 127.13.149.254:38609
I20260924 13:51:27.830528 31481 heartbeater.cc:464] Registering TS with master...
I20260924 13:51:27.830696 31481 heartbeater.cc:511] Master 127.13.149.254:38609 requested a full tablet report, sending...
I20260924 13:51:27.831043 31022 ts_manager.cc:194] Registered new tserver with Master: 4074d292e57d49cabc2e8ccc6a26a4cd (127.13.149.195:39491)
I20260924 13:51:27.831415 31022 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.149.195@KRBTEST.COM'} at 127.13.149.195:41007
I20260924 13:51:27.836966 13911 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260924 13:51:27.843472 31079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:27.838200 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39700 (local address 127.13.149.254:38609)
0924 13:51:27.838238 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0924 13:51:27.838240 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:51:27.838349 (+ 109us) server_negotiation.cc:407] Connection header received
0924 13:51:27.838400 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:51:27.838401 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:51:27.838408 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:51:27.838454 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:51:27.838974 (+ 520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:51:27.839451 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:51:27.840210 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:51:27.840405 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:51:27.840450 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:51:27.841619 (+ 1169us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:51:27.841620 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:51:27.841620 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 13:51:27.841626 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:51:27.842629 (+ 1003us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:51:27.842916 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:51:27.842916 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:51:27.842917 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:51:27.842959 (+ 42us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:51:27.843083 (+ 124us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:51:27.843083 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:51:27.843083 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:51:27.843206 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:51:27.843275 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0924 13:51:27.843350 (+ 75us) server_negotiation.cc:299] Negotiation successful
0924 13:51:27.843410 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260924 13:51:27.844691 31022 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39700:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 13:51:27.846236 31022 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 13:51:27.852175 31490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:27.848284 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.195:39491 (local address 127.0.0.1:59576)
0924 13:51:27.848469 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0924 13:51:27.848477 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0924 13:51:27.848506 (+ 29us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:51:27.848747 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:51:27.848748 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:51:27.848751 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 13:51:27.848806 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:51:27.848807 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:51:27.849835 (+ 1028us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:51:27.849836 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:51:27.851120 (+ 1284us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:51:27.851123 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:51:27.851884 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:51:27.851884 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:51:27.851963 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:51:27.851977 (+ 14us) client_negotiation.cc:769] Sending connection context
0924 13:51:27.852041 (+ 64us) client_negotiation.cc:240] Negotiation successful
0924 13:51:27.852097 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":38,"threads_started":1}
I20260924 13:51:27.852187 31494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:27.848448 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59576 (local address 127.13.149.195:39491)
0924 13:51:27.848611 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0924 13:51:27.848614 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 13:51:27.848622 (+ 8us) server_negotiation.cc:407] Connection header received
0924 13:51:27.848650 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:51:27.848652 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:51:27.848663 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:51:27.848735 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 13:51:27.848889 (+ 154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:51:27.849750 (+ 861us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:51:27.851204 (+ 1454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:51:27.851806 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:51:27.851872 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:51:27.851929 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0924 13:51:27.852061 (+ 132us) server_negotiation.cc:299] Negotiation successful
0924 13:51:27.852101 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135,"thread_start_us":88,"threads_started":1}
I20260924 13:51:27.852514 31492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:27.848113 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.193:39821 (local address 127.0.0.1:41844)
0924 13:51:27.848386 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0924 13:51:27.848395 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0924 13:51:27.848448 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:51:27.848723 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:51:27.848727 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:51:27.848729 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 13:51:27.848796 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:51:27.848798 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:51:27.850097 (+ 1299us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:51:27.850098 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:51:27.851342 (+ 1244us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:51:27.851344 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:51:27.852275 (+ 931us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:51:27.852275 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:51:27.852339 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:51:27.852344 (+ 5us) client_negotiation.cc:769] Sending connection context
0924 13:51:27.852391 (+ 47us) client_negotiation.cc:240] Negotiation successful
0924 13:51:27.852433 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":258,"mutex_wait_us":11,"thread_start_us":53,"threads_started":1}
I20260924 13:51:27.852571 31491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:27.848222 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41844 (local address 127.13.149.193:39821)
0924 13:51:27.848564 (+ 342us) server_negotiation.cc:206] Beginning negotiation
0924 13:51:27.848569 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 13:51:27.848578 (+ 9us) server_negotiation.cc:407] Connection header received
0924 13:51:27.848615 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:51:27.848617 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:51:27.848632 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:51:27.848695 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 13:51:27.848858 (+ 163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:51:27.850009 (+ 1151us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:51:27.851414 (+ 1405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:51:27.852116 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:51:27.852188 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:51:27.852237 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0924 13:51:27.852437 (+ 200us) server_negotiation.cc:299] Negotiation successful
0924 13:51:27.852484 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":305,"thread_start_us":80,"threads_started":1}
I20260924 13:51:27.852607 31493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:27.848218 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34772 (local address 127.13.149.194:35009)
0924 13:51:27.848900 (+ 682us) server_negotiation.cc:206] Beginning negotiation
0924 13:51:27.848906 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0924 13:51:27.848933 (+ 27us) server_negotiation.cc:407] Connection header received
0924 13:51:27.848981 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:51:27.848984 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:51:27.848998 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:51:27.849065 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 13:51:27.849255 (+ 190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:51:27.850271 (+ 1016us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:51:27.851460 (+ 1189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:51:27.852127 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:51:27.852188 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:51:27.852237 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0924 13:51:27.852478 (+ 241us) server_negotiation.cc:299] Negotiation successful
0924 13:51:27.852520 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":585,"thread_start_us":110,"threads_started":1}
I20260924 13:51:27.853003 31489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:27.848111 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.194:35009 (local address 127.0.0.1:34772)
0924 13:51:27.848262 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0924 13:51:27.848325 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0924 13:51:27.848384 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:51:27.849150 (+ 766us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:51:27.849151 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:51:27.849154 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 13:51:27.849190 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:51:27.849192 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:51:27.850365 (+ 1173us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:51:27.850366 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:51:27.851389 (+ 1023us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:51:27.851392 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:51:27.852273 (+ 881us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:51:27.852274 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:51:27.852339 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:51:27.852342 (+ 3us) client_negotiation.cc:769] Sending connection context
0924 13:51:27.852388 (+ 46us) client_negotiation.cc:240] Negotiation successful
0924 13:51:27.852432 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"mutex_wait_us":2,"thread_start_us":52,"threads_started":1}
I20260924 13:51:27.853303 31415 tablet_service.cc:1467] Processing CreateTablet for tablet 12478b38544a40cdaa079854ef68d737 (DEFAULT_TABLE table=test-table [id=e079092c319e477795f5f6db1b2548b1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 13:51:27.853484 31147 tablet_service.cc:1467] Processing CreateTablet for tablet 12478b38544a40cdaa079854ef68d737 (DEFAULT_TABLE table=test-table [id=e079092c319e477795f5f6db1b2548b1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 13:51:27.853667 31415 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 12478b38544a40cdaa079854ef68d737. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 13:51:27.853725 31147 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 12478b38544a40cdaa079854ef68d737. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 13:51:27.853644 31281 tablet_service.cc:1467] Processing CreateTablet for tablet 12478b38544a40cdaa079854ef68d737 (DEFAULT_TABLE table=test-table [id=e079092c319e477795f5f6db1b2548b1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 13:51:27.853937 31281 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 12478b38544a40cdaa079854ef68d737. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 13:51:27.856149 31495 tablet_bootstrap.cc:492] T 12478b38544a40cdaa079854ef68d737 P 27e4fdd555804c8dbeb3d8ef892cf590: Bootstrap starting.
I20260924 13:51:27.856726 31495 tablet_bootstrap.cc:654] T 12478b38544a40cdaa079854ef68d737 P 27e4fdd555804c8dbeb3d8ef892cf590: Neither blocks nor log segments found. Creating new log.
I20260924 13:51:27.856894 31496 tablet_bootstrap.cc:492] T 12478b38544a40cdaa079854ef68d737 P 4074d292e57d49cabc2e8ccc6a26a4cd: Bootstrap starting.
I20260924 13:51:27.857331 31495 log.cc:824] T 12478b38544a40cdaa079854ef68d737 P 27e4fdd555804c8dbeb3d8ef892cf590: Log is configured to *not* fsync() on all Append() calls
I20260924 13:51:27.857554 31496 tablet_bootstrap.cc:654] T 12478b38544a40cdaa079854ef68d737 P 4074d292e57d49cabc2e8ccc6a26a4cd: Neither blocks nor log segments found. Creating new log.
I20260924 13:51:27.858045 31496 log.cc:824] T 12478b38544a40cdaa079854ef68d737 P 4074d292e57d49cabc2e8ccc6a26a4cd: Log is configured to *not* fsync() on all Append() calls
I20260924 13:51:27.858110 31495 tablet_bootstrap.cc:492] T 12478b38544a40cdaa079854ef68d737 P 27e4fdd555804c8dbeb3d8ef892cf590: No bootstrap required, opened a new log
I20260924 13:51:27.858163 31495 ts_tablet_manager.cc:1397] T 12478b38544a40cdaa079854ef68d737 P 27e4fdd555804c8dbeb3d8ef892cf590: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260924 13:51:27.858372 31497 tablet_bootstrap.cc:492] T 12478b38544a40cdaa079854ef68d737 P 446c38483d164f7784169a299e102035: Bootstrap starting.
I20260924 13:51:27.858686 31496 tablet_bootstrap.cc:492] T 12478b38544a40cdaa079854ef68d737 P 4074d292e57d49cabc2e8ccc6a26a4cd: No bootstrap required, opened a new log
I20260924 13:51:27.858747 31496 ts_tablet_manager.cc:1397] T 12478b38544a40cdaa079854ef68d737 P 4074d292e57d49cabc2e8ccc6a26a4cd: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260924 13:51:27.859010 31497 tablet_bootstrap.cc:654] T 12478b38544a40cdaa079854ef68d737 P 446c38483d164f7784169a299e102035: Neither blocks nor log segments found. Creating new log.
I20260924 13:51:27.859535 31497 log.cc:824] T 12478b38544a40cdaa079854ef68d737 P 446c38483d164f7784169a299e102035: Log is configured to *not* fsync() on all Append() calls
I20260924 13:51:27.860174 31496 raft_consensus.cc:348] T 12478b38544a40cdaa079854ef68d737 P 4074d292e57d49cabc2e8ccc6a26a4cd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "446c38483d164f7784169a299e102035" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 35009 } } peers { permanent_uuid: "27e4fdd555804c8dbeb3d8ef892cf590" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 39821 } } peers { permanent_uuid: "4074d292e57d49cabc2e8ccc6a26a4cd" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 39491 } }
I20260924 13:51:27.860139 31495 raft_consensus.cc:348] T 12478b38544a40cdaa079854ef68d737 P 27e4fdd555804c8dbeb3d8ef892cf590 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "446c38483d164f7784169a299e102035" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 35009 } } peers { permanent_uuid: "27e4fdd555804c8dbeb3d8ef892cf590" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 39821 } } peers { permanent_uuid: "4074d292e57d49cabc2e8ccc6a26a4cd" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 39491 } }
I20260924 13:51:27.860294 31496 raft_consensus.cc:374] T 12478b38544a40cdaa079854ef68d737 P 4074d292e57d49cabc2e8ccc6a26a4cd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 13:51:27.860316 31496 raft_consensus.cc:729] T 12478b38544a40cdaa079854ef68d737 P 4074d292e57d49cabc2e8ccc6a26a4cd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4074d292e57d49cabc2e8ccc6a26a4cd, State: Initialized, Role: FOLLOWER
I20260924 13:51:27.860306 31495 raft_consensus.cc:374] T 12478b38544a40cdaa079854ef68d737 P 27e4fdd555804c8dbeb3d8ef892cf590 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 13:51:27.860340 31495 raft_consensus.cc:729] T 12478b38544a40cdaa079854ef68d737 P 27e4fdd555804c8dbeb3d8ef892cf590 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 27e4fdd555804c8dbeb3d8ef892cf590, State: Initialized, Role: FOLLOWER
I20260924 13:51:27.860383 31497 tablet_bootstrap.cc:492] T 12478b38544a40cdaa079854ef68d737 P 446c38483d164f7784169a299e102035: No bootstrap required, opened a new log
I20260924 13:51:27.860432 31497 ts_tablet_manager.cc:1397] T 12478b38544a40cdaa079854ef68d737 P 446c38483d164f7784169a299e102035: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260924 13:51:27.860409 31496 consensus_queue.cc:260] T 12478b38544a40cdaa079854ef68d737 P 4074d292e57d49cabc2e8ccc6a26a4cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "446c38483d164f7784169a299e102035" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 35009 } } peers { permanent_uuid: "27e4fdd555804c8dbeb3d8ef892cf590" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 39821 } } peers { permanent_uuid: "4074d292e57d49cabc2e8ccc6a26a4cd" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 39491 } }
I20260924 13:51:27.860481 31495 consensus_queue.cc:260] T 12478b38544a40cdaa079854ef68d737 P 27e4fdd555804c8dbeb3d8ef892cf590 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "446c38483d164f7784169a299e102035" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 35009 } } peers { permanent_uuid: "27e4fdd555804c8dbeb3d8ef892cf590" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 39821 } } peers { permanent_uuid: "4074d292e57d49cabc2e8ccc6a26a4cd" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 39491 } }
I20260924 13:51:27.860652 31496 ts_tablet_manager.cc:1428] T 12478b38544a40cdaa079854ef68d737 P 4074d292e57d49cabc2e8ccc6a26a4cd: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260924 13:51:27.860663 31481 heartbeater.cc:503] Master 127.13.149.254:38609 was elected leader, sending a full tablet report...
I20260924 13:51:27.860726 31495 ts_tablet_manager.cc:1428] T 12478b38544a40cdaa079854ef68d737 P 27e4fdd555804c8dbeb3d8ef892cf590: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260924 13:51:27.860842 31213 heartbeater.cc:503] Master 127.13.149.254:38609 was elected leader, sending a full tablet report...
I20260924 13:51:27.862123 31497 raft_consensus.cc:348] T 12478b38544a40cdaa079854ef68d737 P 446c38483d164f7784169a299e102035 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "446c38483d164f7784169a299e102035" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 35009 } } peers { permanent_uuid: "27e4fdd555804c8dbeb3d8ef892cf590" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 39821 } } peers { permanent_uuid: "4074d292e57d49cabc2e8ccc6a26a4cd" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 39491 } }
I20260924 13:51:27.862247 31497 raft_consensus.cc:374] T 12478b38544a40cdaa079854ef68d737 P 446c38483d164f7784169a299e102035 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 13:51:27.862278 31497 raft_consensus.cc:729] T 12478b38544a40cdaa079854ef68d737 P 446c38483d164f7784169a299e102035 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 446c38483d164f7784169a299e102035, State: Initialized, Role: FOLLOWER
I20260924 13:51:27.862390 31497 consensus_queue.cc:260] T 12478b38544a40cdaa079854ef68d737 P 446c38483d164f7784169a299e102035 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "446c38483d164f7784169a299e102035" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 35009 } } peers { permanent_uuid: "27e4fdd555804c8dbeb3d8ef892cf590" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 39821 } } peers { permanent_uuid: "4074d292e57d49cabc2e8ccc6a26a4cd" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 39491 } }
I20260924 13:51:27.862610 31497 ts_tablet_manager.cc:1428] T 12478b38544a40cdaa079854ef68d737 P 446c38483d164f7784169a299e102035: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260924 13:51:27.862659 31347 heartbeater.cc:503] Master 127.13.149.254:38609 was elected leader, sending a full tablet report...
I20260924 13:51:27.913950 31501 raft_consensus.cc:482] T 12478b38544a40cdaa079854ef68d737 P 4074d292e57d49cabc2e8ccc6a26a4cd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 13:51:27.914085 31501 raft_consensus.cc:504] T 12478b38544a40cdaa079854ef68d737 P 4074d292e57d49cabc2e8ccc6a26a4cd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "446c38483d164f7784169a299e102035" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 35009 } } peers { permanent_uuid: "27e4fdd555804c8dbeb3d8ef892cf590" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 39821 } } peers { permanent_uuid: "4074d292e57d49cabc2e8ccc6a26a4cd" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 39491 } }
I20260924 13:51:27.914428 31501 leader_election.cc:288] T 12478b38544a40cdaa079854ef68d737 P 4074d292e57d49cabc2e8ccc6a26a4cd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 446c38483d164f7784169a299e102035 (127.13.149.194:35009), 27e4fdd555804c8dbeb3d8ef892cf590 (127.13.149.193:39821)
I20260924 13:51:27.917109 31483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:27.914627 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.193:39821 (local address 127.13.149.195:55627)
0924 13:51:27.914682 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0924 13:51:27.914695 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0924 13:51:27.914753 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:51:27.914876 (+ 123us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:51:27.914877 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:51:27.914878 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 13:51:27.914948 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:51:27.914950 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:51:27.915559 (+ 609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:51:27.915560 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:51:27.916303 (+ 743us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:51:27.916305 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:51:27.916839 (+ 534us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:51:27.916841 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:51:27.916937 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:51:27.916941 (+ 4us) client_negotiation.cc:769] Sending connection context
0924 13:51:27.916990 (+ 49us) client_negotiation.cc:240] Negotiation successful
0924 13:51:27.917032 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260924 13:51:27.917110 31491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:27.914707 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.149.195:55627 (local address 127.13.149.193:39821)
0924 13:51:27.914767 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0924 13:51:27.914769 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:51:27.914777 (+ 8us) server_negotiation.cc:407] Connection header received
0924 13:51:27.914807 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:51:27.914808 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:51:27.914817 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:51:27.914865 (+ 48us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 13:51:27.915011 (+ 146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:51:27.915483 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:51:27.916396 (+ 913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:51:27.916765 (+ 369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:51:27.916827 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:51:27.916837 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0924 13:51:27.916995 (+ 158us) server_negotiation.cc:299] Negotiation successful
0924 13:51:27.917034 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260924 13:51:27.917285 31493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:27.914705 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.149.195:46735 (local address 127.13.149.194:35009)
0924 13:51:27.914758 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0924 13:51:27.914760 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:51:27.914897 (+ 137us) server_negotiation.cc:407] Connection header received
0924 13:51:27.914943 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:51:27.914944 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:51:27.914952 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:51:27.914995 (+ 43us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 13:51:27.915187 (+ 192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:51:27.915646 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:51:27.916553 (+ 907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:51:27.916953 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:51:27.917014 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:51:27.917022 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0924 13:51:27.917162 (+ 140us) server_negotiation.cc:299] Negotiation successful
0924 13:51:27.917211 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260924 13:51:27.917352 31504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:27.914627 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.194:35009 (local address 127.13.149.195:46735)
0924 13:51:27.914826 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0924 13:51:27.914834 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0924 13:51:27.914890 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:51:27.915005 (+ 115us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:51:27.915006 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:51:27.915008 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 13:51:27.915082 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:51:27.915084 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:51:27.915723 (+ 639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:51:27.915724 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:51:27.916467 (+ 743us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:51:27.916469 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:51:27.917023 (+ 554us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:51:27.917024 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:51:27.917098 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:51:27.917102 (+ 4us) client_negotiation.cc:769] Sending connection context
0924 13:51:27.917148 (+ 46us) client_negotiation.cc:240] Negotiation successful
0924 13:51:27.917279 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":183,"mutex_wait_us":14,"thread_start_us":112,"threads_started":1}
I20260924 13:51:27.917701 31301 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "12478b38544a40cdaa079854ef68d737" candidate_uuid: "4074d292e57d49cabc2e8ccc6a26a4cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "446c38483d164f7784169a299e102035" is_pre_election: true
I20260924 13:51:27.917697 31167 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "12478b38544a40cdaa079854ef68d737" candidate_uuid: "4074d292e57d49cabc2e8ccc6a26a4cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "27e4fdd555804c8dbeb3d8ef892cf590" is_pre_election: true
I20260924 13:51:27.917873 31167 raft_consensus.cc:2454] T 12478b38544a40cdaa079854ef68d737 P 27e4fdd555804c8dbeb3d8ef892cf590 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4074d292e57d49cabc2e8ccc6a26a4cd in term 0.
I20260924 13:51:27.917873 31301 raft_consensus.cc:2454] T 12478b38544a40cdaa079854ef68d737 P 446c38483d164f7784169a299e102035 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4074d292e57d49cabc2e8ccc6a26a4cd in term 0.
I20260924 13:51:27.918105 31368 leader_election.cc:302] T 12478b38544a40cdaa079854ef68d737 P 4074d292e57d49cabc2e8ccc6a26a4cd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4074d292e57d49cabc2e8ccc6a26a4cd, 446c38483d164f7784169a299e102035; no voters:
I20260924 13:51:27.918231 31501 raft_consensus.cc:2781] T 12478b38544a40cdaa079854ef68d737 P 4074d292e57d49cabc2e8ccc6a26a4cd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 13:51:27.918293 31501 raft_consensus.cc:482] T 12478b38544a40cdaa079854ef68d737 P 4074d292e57d49cabc2e8ccc6a26a4cd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 13:51:27.918309 31501 raft_consensus.cc:3037] T 12478b38544a40cdaa079854ef68d737 P 4074d292e57d49cabc2e8ccc6a26a4cd [term 0 FOLLOWER]: Advancing to term 1
I20260924 13:51:27.918978 31501 raft_consensus.cc:504] T 12478b38544a40cdaa079854ef68d737 P 4074d292e57d49cabc2e8ccc6a26a4cd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "446c38483d164f7784169a299e102035" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 35009 } } peers { permanent_uuid: "27e4fdd555804c8dbeb3d8ef892cf590" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 39821 } } peers { permanent_uuid: "4074d292e57d49cabc2e8ccc6a26a4cd" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 39491 } }
I20260924 13:51:27.919106 31501 leader_election.cc:288] T 12478b38544a40cdaa079854ef68d737 P 4074d292e57d49cabc2e8ccc6a26a4cd [CANDIDATE]: Term 1 election: Requested vote from peers 446c38483d164f7784169a299e102035 (127.13.149.194:35009), 27e4fdd555804c8dbeb3d8ef892cf590 (127.13.149.193:39821)
I20260924 13:51:27.919258 31301 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "12478b38544a40cdaa079854ef68d737" candidate_uuid: "4074d292e57d49cabc2e8ccc6a26a4cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "446c38483d164f7784169a299e102035"
I20260924 13:51:27.919265 31167 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "12478b38544a40cdaa079854ef68d737" candidate_uuid: "4074d292e57d49cabc2e8ccc6a26a4cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "27e4fdd555804c8dbeb3d8ef892cf590"
I20260924 13:51:27.919342 31301 raft_consensus.cc:3037] T 12478b38544a40cdaa079854ef68d737 P 446c38483d164f7784169a299e102035 [term 0 FOLLOWER]: Advancing to term 1
I20260924 13:51:27.919356 31167 raft_consensus.cc:3037] T 12478b38544a40cdaa079854ef68d737 P 27e4fdd555804c8dbeb3d8ef892cf590 [term 0 FOLLOWER]: Advancing to term 1
I20260924 13:51:27.920047 31167 raft_consensus.cc:2454] T 12478b38544a40cdaa079854ef68d737 P 27e4fdd555804c8dbeb3d8ef892cf590 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4074d292e57d49cabc2e8ccc6a26a4cd in term 1.
I20260924 13:51:27.920050 31301 raft_consensus.cc:2454] T 12478b38544a40cdaa079854ef68d737 P 446c38483d164f7784169a299e102035 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4074d292e57d49cabc2e8ccc6a26a4cd in term 1.
I20260924 13:51:27.920225 31368 leader_election.cc:302] T 12478b38544a40cdaa079854ef68d737 P 4074d292e57d49cabc2e8ccc6a26a4cd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4074d292e57d49cabc2e8ccc6a26a4cd, 446c38483d164f7784169a299e102035; no voters:
I20260924 13:51:27.920312 31501 raft_consensus.cc:2781] T 12478b38544a40cdaa079854ef68d737 P 4074d292e57d49cabc2e8ccc6a26a4cd [term 1 FOLLOWER]: Leader election won for term 1
I20260924 13:51:27.920457 31501 raft_consensus.cc:686] T 12478b38544a40cdaa079854ef68d737 P 4074d292e57d49cabc2e8ccc6a26a4cd [term 1 LEADER]: Becoming Leader. State: Replica: 4074d292e57d49cabc2e8ccc6a26a4cd, State: Running, Role: LEADER
I20260924 13:51:27.920544 31501 consensus_queue.cc:237] T 12478b38544a40cdaa079854ef68d737 P 4074d292e57d49cabc2e8ccc6a26a4cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "446c38483d164f7784169a299e102035" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 35009 } } peers { permanent_uuid: "27e4fdd555804c8dbeb3d8ef892cf590" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 39821 } } peers { permanent_uuid: "4074d292e57d49cabc2e8ccc6a26a4cd" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 39491 } }
I20260924 13:51:27.921345 31021 catalog_manager.cc:5495] T 12478b38544a40cdaa079854ef68d737 P 4074d292e57d49cabc2e8ccc6a26a4cd reported cstate change: term changed from 0 to 1, leader changed from <none> to 4074d292e57d49cabc2e8ccc6a26a4cd (127.13.149.195). New cstate: current_term: 1 leader_uuid: "4074d292e57d49cabc2e8ccc6a26a4cd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "446c38483d164f7784169a299e102035" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 35009 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "27e4fdd555804c8dbeb3d8ef892cf590" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 39821 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4074d292e57d49cabc2e8ccc6a26a4cd" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 39491 } health_report { overall_health: HEALTHY } } }
W20260924 13:51:27.923619 31348 tablet.cc:2369] T 12478b38544a40cdaa079854ef68d737 P 446c38483d164f7784169a299e102035: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260924 13:51:27.928421 31494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:27.926218 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59588 (local address 127.13.149.195:39491)
0924 13:51:27.926261 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0924 13:51:27.926263 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:51:27.926272 (+ 9us) server_negotiation.cc:407] Connection header received
0924 13:51:27.926341 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:51:27.926342 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:51:27.926359 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:51:27.926412 (+ 53us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 13:51:27.926515 (+ 103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:51:27.926989 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:51:27.927742 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:51:27.927929 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:51:27.927982 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:51:27.928118 (+ 136us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 13:51:27.928211 (+ 93us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 13:51:27.928280 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0924 13:51:27.928314 (+ 34us) server_negotiation.cc:299] Negotiation successful
0924 13:51:27.928333 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260924 13:51:27.930008 31301 raft_consensus.cc:1261] T 12478b38544a40cdaa079854ef68d737 P 446c38483d164f7784169a299e102035 [term 1 FOLLOWER]: Refusing update from remote peer 4074d292e57d49cabc2e8ccc6a26a4cd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 13:51:27.930060 31167 raft_consensus.cc:1261] T 12478b38544a40cdaa079854ef68d737 P 27e4fdd555804c8dbeb3d8ef892cf590 [term 1 FOLLOWER]: Refusing update from remote peer 4074d292e57d49cabc2e8ccc6a26a4cd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 13:51:27.930322 31505 consensus_queue.cc:1035] T 12478b38544a40cdaa079854ef68d737 P 4074d292e57d49cabc2e8ccc6a26a4cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "446c38483d164f7784169a299e102035" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 35009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 13:51:27.930437 31501 consensus_queue.cc:1035] T 12478b38544a40cdaa079854ef68d737 P 4074d292e57d49cabc2e8ccc6a26a4cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "27e4fdd555804c8dbeb3d8ef892cf590" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 39821 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 13:51:27.932436 31508 mvcc.cc:205] Tried to move back new op lower bound from 7332896308959223808 to 7332896308923224064. Current Snapshot: MvccSnapshot[applied={T|T < 7332896308959223808}]
I20260924 13:51:27.932868 31509 mvcc.cc:205] Tried to move back new op lower bound from 7332896308959223808 to 7332896308923224064. Current Snapshot: MvccSnapshot[applied={T|T < 7332896308959223808}]
I20260924 13:51:27.933328 31510 mvcc.cc:205] Tried to move back new op lower bound from 7332896308959223808 to 7332896308923224064. Current Snapshot: MvccSnapshot[applied={T|T < 7332896308959223808}]
I20260924 13:51:27.934394 31021 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39700:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 13:51:27.934490 31021 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39700:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 13:51:27.935173 31021 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3917537cb6e24ee18ec0992c6830480c: Sending DeleteTablet for 3 replicas of tablet 12478b38544a40cdaa079854ef68d737
I20260924 13:51:27.935509 31147 tablet_service.cc:1514] Processing DeleteTablet for tablet 12478b38544a40cdaa079854ef68d737 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 13:51:27 UTC) from {username='kudu', principal='kudu/127.13.149.254@KRBTEST.COM'} at 127.0.0.1:41844
I20260924 13:51:27.935525 31415 tablet_service.cc:1514] Processing DeleteTablet for tablet 12478b38544a40cdaa079854ef68d737 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 13:51:27 UTC) from {username='kudu', principal='kudu/127.13.149.254@KRBTEST.COM'} at 127.0.0.1:59576
I20260924 13:51:27.935659 31281 tablet_service.cc:1514] Processing DeleteTablet for tablet 12478b38544a40cdaa079854ef68d737 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 13:51:27 UTC) from {username='kudu', principal='kudu/127.13.149.254@KRBTEST.COM'} at 127.0.0.1:34772
I20260924 13:51:27.935721 31517 tablet_replica.cc:330] T 12478b38544a40cdaa079854ef68d737 P 27e4fdd555804c8dbeb3d8ef892cf590: stopping tablet replica
I20260924 13:51:27.935803 31517 raft_consensus.cc:2229] T 12478b38544a40cdaa079854ef68d737 P 27e4fdd555804c8dbeb3d8ef892cf590 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 13:51:27.935889 31517 raft_consensus.cc:2258] T 12478b38544a40cdaa079854ef68d737 P 27e4fdd555804c8dbeb3d8ef892cf590 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 13:51:27.936221 13911 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJN9WRF/build/release/bin/kudu with pid 31084
I20260924 13:51:27.937454 31519 tablet_replica.cc:330] T 12478b38544a40cdaa079854ef68d737 P 446c38483d164f7784169a299e102035: stopping tablet replica
I20260924 13:51:27.937549 31519 raft_consensus.cc:2229] T 12478b38544a40cdaa079854ef68d737 P 446c38483d164f7784169a299e102035 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 13:51:27.937645 31519 raft_consensus.cc:2258] T 12478b38544a40cdaa079854ef68d737 P 446c38483d164f7784169a299e102035 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 13:51:27.937780 31518 tablet_replica.cc:330] T 12478b38544a40cdaa079854ef68d737 P 4074d292e57d49cabc2e8ccc6a26a4cd: stopping tablet replica
I20260924 13:51:27.937861 31518 raft_consensus.cc:2229] T 12478b38544a40cdaa079854ef68d737 P 4074d292e57d49cabc2e8ccc6a26a4cd [term 1 LEADER]: Raft consensus shutting down.
I20260924 13:51:27.937981 31518 raft_consensus.cc:2258] T 12478b38544a40cdaa079854ef68d737 P 4074d292e57d49cabc2e8ccc6a26a4cd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 13:51:27.938194 31519 ts_tablet_manager.cc:1905] T 12478b38544a40cdaa079854ef68d737 P 446c38483d164f7784169a299e102035: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 13:51:27.938334 31518 ts_tablet_manager.cc:1905] T 12478b38544a40cdaa079854ef68d737 P 4074d292e57d49cabc2e8ccc6a26a4cd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 13:51:27.939997 31518 ts_tablet_manager.cc:1918] T 12478b38544a40cdaa079854ef68d737 P 4074d292e57d49cabc2e8ccc6a26a4cd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 13:51:27.939996 31519 ts_tablet_manager.cc:1918] T 12478b38544a40cdaa079854ef68d737 P 446c38483d164f7784169a299e102035: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 13:51:27.940068 31518 log.cc:1196] T 12478b38544a40cdaa079854ef68d737 P 4074d292e57d49cabc2e8ccc6a26a4cd: Deleting WAL directory at /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790257629041645-13911-0/minicluster-data/ts-2/wal/wals/12478b38544a40cdaa079854ef68d737
I20260924 13:51:27.940068 31519 log.cc:1196] T 12478b38544a40cdaa079854ef68d737 P 446c38483d164f7784169a299e102035: Deleting WAL directory at /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790257629041645-13911-0/minicluster-data/ts-1/wal/wals/12478b38544a40cdaa079854ef68d737
I20260924 13:51:27.940358 31518 ts_tablet_manager.cc:1939] T 12478b38544a40cdaa079854ef68d737 P 4074d292e57d49cabc2e8ccc6a26a4cd: Deleting consensus metadata
I20260924 13:51:27.940387 31519 ts_tablet_manager.cc:1939] T 12478b38544a40cdaa079854ef68d737 P 446c38483d164f7784169a299e102035: Deleting consensus metadata
I20260924 13:51:27.940761 31006 catalog_manager.cc:4841] TS 446c38483d164f7784169a299e102035 (127.13.149.194:35009): tablet 12478b38544a40cdaa079854ef68d737 (table test-table [id=e079092c319e477795f5f6db1b2548b1]) successfully deleted
I20260924 13:51:27.940822 31008 catalog_manager.cc:4841] TS 4074d292e57d49cabc2e8ccc6a26a4cd (127.13.149.195:39491): tablet 12478b38544a40cdaa079854ef68d737 (table test-table [id=e079092c319e477795f5f6db1b2548b1]) successfully deleted
W20260924 13:51:27.941586 31008 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.13.149.193:39821 (error 108)
I20260924 13:51:27.941707 13911 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJN9WRF/build/release/bin/kudu with pid 31218
I20260924 13:51:27.942580 31489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:27.942458 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.193:39821 (local address 127.0.0.1:41856)
0924 13:51:27.942509 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0924 13:51:27.942547 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.149.193:39821: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260924 13:51:27.942709 31008 catalog_manager.cc:4584] TS 27e4fdd555804c8dbeb3d8ef892cf590 (127.13.149.193:39821): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 12478b38544a40cdaa079854ef68d737: Network error: Client connection negotiation failed: client connection to 127.13.149.193:39821: connect: Connection refused (error 111)
I20260924 13:51:27.946946 13911 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJN9WRF/build/release/bin/kudu with pid 31352
I20260924 13:51:27.951380 13911 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJN9WRF/build/release/bin/kudu with pid 30990
2026-09-24T13:51:27Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (1815 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-09-24T13:51:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T13:51:27Z Disabled control of system clock
I20260924 13:51:27.976707 13911 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJN9WRF/build/release/bin/kudu
/tmp/dist-test-taskJN9WRF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790257629041645-13911-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790257629041645-13911-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790257629041645-13911-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790257629041645-13911-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.149.254:44727
--webserver_interface=127.13.149.254
--webserver_port=0
--builtin_ntp_servers=127.13.149.212:43381
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.149.254:44727
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260924 13:51:28.055727 31524 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 13:51:28.055944 31524 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 13:51:28.055964 31524 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 13:51:28.057080 31524 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 13:51:28.057125 31524 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 13:51:28.057137 31524 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 13:51:28.057152 31524 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 13:51:28.057161 31524 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 13:51:28.058461 31524 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.149.212:43381
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790257629041645-13911-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790257629041645-13911-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.149.254:44727
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.149.254:44727
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790257629041645-13911-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.149.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31524
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790257629041645-13911-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 12:29:37 UTC on ccycloud.quasar-iqzbxn.root.comops.site
build id 4670106
I20260924 13:51:28.058661 31524 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 13:51:28.058899 31524 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 13:51:28.063575 31530 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 13:51:28.063694 31524 server_base.cc:1034] running on GCE node
W20260924 13:51:28.063565 31529 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 13:51:28.063565 31532 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 13:51:28.064057 31524 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 13:51:28.064347 31524 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 13:51:28.065495 31524 hybrid_clock.cc:648] HybridClock initialized: now 1790257888065480 us; error 30 us; skew 500 ppm
I20260924 13:51:28.066707 31524 webserver.cc:460] Webserver started at http://127.13.149.254:43639/ using document root <none> and password file <none>
I20260924 13:51:28.066934 31524 fs_manager.cc:359] Metadata directory not provided
I20260924 13:51:28.066978 31524 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 13:51:28.067101 31524 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 13:51:28.068141 31524 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790257629041645-13911-0/minicluster-data/master-0/data/instance:
uuid: "3c21785cfa5c42e28cbc40ea271097c5"
format_stamp: "Formatted at 2026-09-24 13:51:28 on dist-test-slave-sjl2"
server_key: "5684c8ab87848e07e627cb9915beae10"
server_key_iv: "e7105af84a4a79b747bf5c5142c5c23d"
server_key_version: "encryptionkey@0"
I20260924 13:51:28.068463 31524 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790257629041645-13911-0/minicluster-data/master-0/wal/instance:
uuid: "3c21785cfa5c42e28cbc40ea271097c5"
format_stamp: "Formatted at 2026-09-24 13:51:28 on dist-test-slave-sjl2"
server_key: "5684c8ab87848e07e627cb9915beae10"
server_key_iv: "e7105af84a4a79b747bf5c5142c5c23d"
server_key_version: "encryptionkey@0"
I20260924 13:51:28.069701 31524 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260924 13:51:28.070330 31538 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 13:51:28.070477 31524 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 13:51:28.070572 31524 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790257629041645-13911-0/minicluster-data/master-0/data,/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790257629041645-13911-0/minicluster-data/master-0/wal
uuid: "3c21785cfa5c42e28cbc40ea271097c5"
format_stamp: "Formatted at 2026-09-24 13:51:28 on dist-test-slave-sjl2"
server_key: "5684c8ab87848e07e627cb9915beae10"
server_key_iv: "e7105af84a4a79b747bf5c5142c5c23d"
server_key_version: "encryptionkey@0"
I20260924 13:51:28.070650 31524 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790257629041645-13911-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790257629041645-13911-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790257629041645-13911-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 13:51:28.094498 31524 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 13:51:28.094830 31524 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 13:51:28.094966 31524 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 13:51:28.098770 31524 rpc_server.cc:307] RPC server started. Bound to: 127.13.149.254:44727
I20260924 13:51:28.098809 31590 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.149.254:44727 every 8 connection(s)
I20260924 13:51:28.099133 31524 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790257629041645-13911-0/minicluster-data/master-0/data/info.pb
I20260924 13:51:28.100051 31591 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 13:51:28.100872 13911 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJN9WRF/build/release/bin/kudu as pid 31524
I20260924 13:51:28.100991 13911 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790257629041645-13911-0/minicluster-data/master-0/wal/instance
I20260924 13:51:28.101217 13911 external_mini_cluster.cc:1428] Setting key 7caee281adaea42dcc0de1b33f94843a
I20260924 13:51:28.103139 31591 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3c21785cfa5c42e28cbc40ea271097c5: Bootstrap starting.
I20260924 13:51:28.103696 31591 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3c21785cfa5c42e28cbc40ea271097c5: Neither blocks nor log segments found. Creating new log.
I20260924 13:51:28.103945 31591 log.cc:824] T 00000000000000000000000000000000 P 3c21785cfa5c42e28cbc40ea271097c5: Log is configured to *not* fsync() on all Append() calls
I20260924 13:51:28.104544 31591 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3c21785cfa5c42e28cbc40ea271097c5: No bootstrap required, opened a new log
I20260924 13:51:28.106385 31594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:28.101760 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51432 (local address 127.13.149.254:44727)
0924 13:51:28.101945 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0924 13:51:28.101952 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0924 13:51:28.101976 (+ 24us) server_negotiation.cc:407] Connection header received
0924 13:51:28.102385 (+ 409us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:51:28.102391 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:51:28.102448 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:51:28.102556 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:51:28.102834 (+ 278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:51:28.103740 (+ 906us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:51:28.104537 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:51:28.105022 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:51:28.105133 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:51:28.105794 (+ 661us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:51:28.105799 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:51:28.105799 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 13:51:28.105959 (+ 160us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:51:28.105990 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 13:51:28.106002 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:51:28.106054 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0924 13:51:28.106138 (+ 84us) server_negotiation.cc:299] Negotiation successful
0924 13:51:28.106218 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":114,"thread_start_us":63,"threads_started":1}
I20260924 13:51:28.106475 31591 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3c21785cfa5c42e28cbc40ea271097c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c21785cfa5c42e28cbc40ea271097c5" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 44727 } }
I20260924 13:51:28.106647 31591 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3c21785cfa5c42e28cbc40ea271097c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 13:51:28.106688 31591 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3c21785cfa5c42e28cbc40ea271097c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3c21785cfa5c42e28cbc40ea271097c5, State: Initialized, Role: FOLLOWER
I20260924 13:51:28.106819 31591 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3c21785cfa5c42e28cbc40ea271097c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c21785cfa5c42e28cbc40ea271097c5" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 44727 } }
I20260924 13:51:28.106899 31591 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3c21785cfa5c42e28cbc40ea271097c5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 13:51:28.106930 31591 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3c21785cfa5c42e28cbc40ea271097c5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 13:51:28.106976 31591 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3c21785cfa5c42e28cbc40ea271097c5 [term 0 FOLLOWER]: Advancing to term 1
I20260924 13:51:28.107486 31591 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3c21785cfa5c42e28cbc40ea271097c5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c21785cfa5c42e28cbc40ea271097c5" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 44727 } }
I20260924 13:51:28.107576 31591 leader_election.cc:302] T 00000000000000000000000000000000 P 3c21785cfa5c42e28cbc40ea271097c5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3c21785cfa5c42e28cbc40ea271097c5; no voters:
I20260924 13:51:28.107729 31591 leader_election.cc:288] T 00000000000000000000000000000000 P 3c21785cfa5c42e28cbc40ea271097c5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 13:51:28.107764 31596 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3c21785cfa5c42e28cbc40ea271097c5 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 13:51:28.107913 31596 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3c21785cfa5c42e28cbc40ea271097c5 [term 1 LEADER]: Becoming Leader. State: Replica: 3c21785cfa5c42e28cbc40ea271097c5, State: Running, Role: LEADER
I20260924 13:51:28.107959 31591 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3c21785cfa5c42e28cbc40ea271097c5 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 13:51:28.107998 31596 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3c21785cfa5c42e28cbc40ea271097c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c21785cfa5c42e28cbc40ea271097c5" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 44727 } }
I20260924 13:51:28.108330 31597 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3c21785cfa5c42e28cbc40ea271097c5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3c21785cfa5c42e28cbc40ea271097c5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c21785cfa5c42e28cbc40ea271097c5" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 44727 } } }
I20260924 13:51:28.108347 31598 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3c21785cfa5c42e28cbc40ea271097c5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3c21785cfa5c42e28cbc40ea271097c5. Latest consensus state: current_term: 1 leader_uuid: "3c21785cfa5c42e28cbc40ea271097c5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c21785cfa5c42e28cbc40ea271097c5" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 44727 } } }
I20260924 13:51:28.108552 31598 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3c21785cfa5c42e28cbc40ea271097c5 [sys.catalog]: This master's current role is: LEADER
I20260924 13:51:28.108467 31597 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3c21785cfa5c42e28cbc40ea271097c5 [sys.catalog]: This master's current role is: LEADER
I20260924 13:51:28.108961 31605 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 13:51:28.109400 31605 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 13:51:28.110731 31605 catalog_manager.cc:1347] Generated new cluster ID: a45304b8379f484e8201f23747bd4800
I20260924 13:51:28.110782 31605 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 13:51:28.129175 31605 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 13:51:28.129714 31605 catalog_manager.cc:1504] Loading token signing keys...
I20260924 13:51:28.131847 31605 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3c21785cfa5c42e28cbc40ea271097c5: Generated new TSK 0
I20260924 13:51:28.132023 31605 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260924 13:51:28.141916 13911 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJN9WRF/build/release/bin/kudu
/tmp/dist-test-taskJN9WRF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790257629041645-13911-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790257629041645-13911-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790257629041645-13911-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790257629041645-13911-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.149.193:0
--local_ip_for_outbound_sockets=127.13.149.193
--webserver_interface=127.13.149.193
--webserver_port=0
--tserver_master_addrs=127.13.149.254:44727
--builtin_ntp_servers=127.13.149.212:43381
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260924 13:51:28.222790 31615 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 13:51:28.223042 31615 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 13:51:28.223075 31615 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 13:51:28.224301 31615 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 13:51:28.224344 31615 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 13:51:28.224380 31615 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.149.193
I20260924 13:51:28.225847 31615 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.149.212:43381
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790257629041645-13911-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790257629041645-13911-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.149.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790257629041645-13911-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.149.193
--webserver_port=0
--tserver_master_addrs=127.13.149.254:44727
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31615
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.149.193
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790257629041645-13911-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 12:29:37 UTC on ccycloud.quasar-iqzbxn.root.comops.site
build id 4670106
I20260924 13:51:28.226099 31615 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 13:51:28.226393 31615 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 13:51:28.231346 31623 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 13:51:28.231313 31620 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 13:51:28.231360 31621 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 13:51:28.231933 31615 server_base.cc:1034] running on GCE node
I20260924 13:51:28.232132 31615 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 13:51:28.232383 31615 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 13:51:28.233529 31615 hybrid_clock.cc:648] HybridClock initialized: now 1790257888233519 us; error 26 us; skew 500 ppm
I20260924 13:51:28.234647 31615 webserver.cc:460] Webserver started at http://127.13.149.193:34651/ using document root <none> and password file <none>
I20260924 13:51:28.234889 31615 fs_manager.cc:359] Metadata directory not provided
I20260924 13:51:28.234931 31615 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 13:51:28.235055 31615 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 13:51:28.236109 31615 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790257629041645-13911-0/minicluster-data/ts-0/data/instance:
uuid: "14188b1e6ba344399d4ed134b6c06e1a"
format_stamp: "Formatted at 2026-09-24 13:51:28 on dist-test-slave-sjl2"
server_key: "2268ddb6552fee32e0c08feae026be38"
server_key_iv: "2cf40c551d204443d2eb72f57f30b25e"
server_key_version: "encryptionkey@0"
I20260924 13:51:28.236423 31615 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790257629041645-13911-0/minicluster-data/ts-0/wal/instance:
uuid: "14188b1e6ba344399d4ed134b6c06e1a"
format_stamp: "Formatted at 2026-09-24 13:51:28 on dist-test-slave-sjl2"
server_key: "2268ddb6552fee32e0c08feae026be38"
server_key_iv: "2cf40c551d204443d2eb72f57f30b25e"
server_key_version: "encryptionkey@0"
I20260924 13:51:28.237665 31615 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260924 13:51:28.238345 31629 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 13:51:28.238477 31615 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 13:51:28.238556 31615 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790257629041645-13911-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790257629041645-13911-0/minicluster-data/ts-0/wal
uuid: "14188b1e6ba344399d4ed134b6c06e1a"
format_stamp: "Formatted at 2026-09-24 13:51:28 on dist-test-slave-sjl2"
server_key: "2268ddb6552fee32e0c08feae026be38"
server_key_iv: "2cf40c551d204443d2eb72f57f30b25e"
server_key_version: "encryptionkey@0"
I20260924 13:51:28.238618 31615 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790257629041645-13911-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790257629041645-13911-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790257629041645-13911-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 13:51:28.247066 31615 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 13:51:28.247344 31615 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 13:51:28.247467 31615 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 13:51:28.247663 31615 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 13:51:28.247963 31615 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 13:51:28.247996 31615 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 13:51:28.248025 31615 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 13:51:28.248049 31615 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 13:51:28.253568 31615 rpc_server.cc:307] RPC server started. Bound to: 127.13.149.193:41703
I20260924 13:51:28.253609 31742 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.149.193:41703 every 8 connection(s)
I20260924 13:51:28.253924 31615 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790257629041645-13911-0/minicluster-data/ts-0/data/info.pb
I20260924 13:51:28.256834 13911 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJN9WRF/build/release/bin/kudu as pid 31615
I20260924 13:51:28.256942 13911 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790257629041645-13911-0/minicluster-data/ts-0/wal/instance
I20260924 13:51:28.257110 13911 external_mini_cluster.cc:1428] Setting key 0842f79c7f05c418caeaa5c0ca0c9412
I20260924 13:51:28.257993 13911 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJN9WRF/build/release/bin/kudu
/tmp/dist-test-taskJN9WRF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790257629041645-13911-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790257629041645-13911-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790257629041645-13911-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790257629041645-13911-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.149.194:0
--local_ip_for_outbound_sockets=127.13.149.194
--webserver_interface=127.13.149.194
--webserver_port=0
--tserver_master_addrs=127.13.149.254:44727
--builtin_ntp_servers=127.13.149.212:43381
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260924 13:51:28.259330 31594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:28.254606 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.149.193:46363 (local address 127.13.149.254:44727)
0924 13:51:28.254659 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0924 13:51:28.254661 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:51:28.254874 (+ 213us) server_negotiation.cc:407] Connection header received
0924 13:51:28.255391 (+ 517us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:51:28.255392 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:51:28.255409 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:51:28.255459 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:51:28.256138 (+ 679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:51:28.256652 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:51:28.257950 (+ 1298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:51:28.258178 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:51:28.258236 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:51:28.259083 (+ 847us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:51:28.259084 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:51:28.259084 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 13:51:28.259089 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:51:28.259105 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 13:51:28.259109 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:51:28.259157 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0924 13:51:28.259243 (+ 86us) server_negotiation.cc:299] Negotiation successful
0924 13:51:28.259268 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260924 13:51:28.259625 31745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:28.254580 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:44727 (local address 127.13.149.193:46363)
0924 13:51:28.254747 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0924 13:51:28.254782 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0924 13:51:28.255291 (+ 509us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:51:28.255537 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:51:28.255542 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:51:28.255551 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0924 13:51:28.256048 (+ 497us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:51:28.256053 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:51:28.256743 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:51:28.256744 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:51:28.257877 (+ 1133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:51:28.257879 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:51:28.258245 (+ 366us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:51:28.258247 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:51:28.258386 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:51:28.258897 (+ 511us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0924 13:51:28.258898 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0924 13:51:28.258940 (+ 42us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0924 13:51:28.258970 (+ 30us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 13:51:28.259171 (+ 201us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 13:51:28.259176 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 13:51:28.259179 (+ 3us) client_negotiation.cc:769] Sending connection context
0924 13:51:28.259244 (+ 65us) client_negotiation.cc:240] Negotiation successful
0924 13:51:28.259354 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":73,"threads_started":1}
I20260924 13:51:28.260166 31743 heartbeater.cc:347] Connected to a master server at 127.13.149.254:44727
I20260924 13:51:28.260263 31743 heartbeater.cc:464] Registering TS with master...
I20260924 13:51:28.260448 31743 heartbeater.cc:511] Master 127.13.149.254:44727 requested a full tablet report, sending...
I20260924 13:51:28.260975 31555 ts_manager.cc:194] Registered new tserver with Master: 14188b1e6ba344399d4ed134b6c06e1a (127.13.149.193:41703)
I20260924 13:51:28.261627 31555 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.13.149.193:46363
W20260924 13:51:28.337855 31746 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 13:51:28.338063 31746 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 13:51:28.338085 31746 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 13:51:28.339291 31746 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 13:51:28.339337 31746 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 13:51:28.339361 31746 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.149.194
I20260924 13:51:28.340683 31746 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.149.212:43381
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790257629041645-13911-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790257629041645-13911-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.149.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790257629041645-13911-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.149.194
--webserver_port=0
--tserver_master_addrs=127.13.149.254:44727
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31746
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.149.194
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790257629041645-13911-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 12:29:37 UTC on ccycloud.quasar-iqzbxn.root.comops.site
build id 4670106
I20260924 13:51:28.340906 31746 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 13:51:28.341200 31746 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 13:51:28.346001 31754 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 13:51:28.346050 31746 server_base.cc:1034] running on GCE node
W20260924 13:51:28.346016 31752 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 13:51:28.346036 31751 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 13:51:28.346446 31746 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 13:51:28.346657 31746 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 13:51:28.347791 31746 hybrid_clock.cc:648] HybridClock initialized: now 1790257888347777 us; error 30 us; skew 500 ppm
I20260924 13:51:28.349063 31746 webserver.cc:460] Webserver started at http://127.13.149.194:33561/ using document root <none> and password file <none>
I20260924 13:51:28.349280 31746 fs_manager.cc:359] Metadata directory not provided
I20260924 13:51:28.349344 31746 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 13:51:28.349468 31746 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 13:51:28.350524 31746 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790257629041645-13911-0/minicluster-data/ts-1/data/instance:
uuid: "60d2ac92d21b4953b0ce338030cfa553"
format_stamp: "Formatted at 2026-09-24 13:51:28 on dist-test-slave-sjl2"
server_key: "b4bf0a932f584dafee53d4403b94ef79"
server_key_iv: "c6f47ba4d7b67d1530e38acf49552a1a"
server_key_version: "encryptionkey@0"
I20260924 13:51:28.350852 31746 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790257629041645-13911-0/minicluster-data/ts-1/wal/instance:
uuid: "60d2ac92d21b4953b0ce338030cfa553"
format_stamp: "Formatted at 2026-09-24 13:51:28 on dist-test-slave-sjl2"
server_key: "b4bf0a932f584dafee53d4403b94ef79"
server_key_iv: "c6f47ba4d7b67d1530e38acf49552a1a"
server_key_version: "encryptionkey@0"
I20260924 13:51:28.352041 31746 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260924 13:51:28.352721 31760 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 13:51:28.352859 31746 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 13:51:28.352955 31746 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790257629041645-13911-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790257629041645-13911-0/minicluster-data/ts-1/wal
uuid: "60d2ac92d21b4953b0ce338030cfa553"
format_stamp: "Formatted at 2026-09-24 13:51:28 on dist-test-slave-sjl2"
server_key: "b4bf0a932f584dafee53d4403b94ef79"
server_key_iv: "c6f47ba4d7b67d1530e38acf49552a1a"
server_key_version: "encryptionkey@0"
I20260924 13:51:28.353025 31746 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790257629041645-13911-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790257629041645-13911-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790257629041645-13911-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 13:51:28.365995 31746 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 13:51:28.366312 31746 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 13:51:28.366434 31746 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 13:51:28.366654 31746 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 13:51:28.366976 31746 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 13:51:28.367009 31746 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 13:51:28.367033 31746 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 13:51:28.367056 31746 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 13:51:28.372509 31746 rpc_server.cc:307] RPC server started. Bound to: 127.13.149.194:36153
I20260924 13:51:28.372560 31873 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.149.194:36153 every 8 connection(s)
I20260924 13:51:28.372867 31746 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790257629041645-13911-0/minicluster-data/ts-1/data/info.pb
I20260924 13:51:28.377981 31594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:28.373627 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.149.194:37681 (local address 127.13.149.254:44727)
0924 13:51:28.373674 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0924 13:51:28.373676 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:51:28.373926 (+ 250us) server_negotiation.cc:407] Connection header received
0924 13:51:28.374388 (+ 462us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:51:28.374390 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:51:28.374400 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:51:28.374451 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:51:28.375174 (+ 723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:51:28.375680 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:51:28.376615 (+ 935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:51:28.376786 (+ 171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:51:28.376835 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:51:28.377764 (+ 929us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:51:28.377765 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:51:28.377766 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 13:51:28.377770 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:51:28.377786 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 13:51:28.377789 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:51:28.377832 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0924 13:51:28.377886 (+ 54us) server_negotiation.cc:299] Negotiation successful
0924 13:51:28.377916 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260924 13:51:28.378213 31876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:28.373638 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:44727 (local address 127.13.149.194:37681)
0924 13:51:28.373810 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0924 13:51:28.373844 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0924 13:51:28.374312 (+ 468us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:51:28.374516 (+ 204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:51:28.374521 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:51:28.374531 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0924 13:51:28.375081 (+ 550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:51:28.375086 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:51:28.375749 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:51:28.375750 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:51:28.376554 (+ 804us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:51:28.376556 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:51:28.376880 (+ 324us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:51:28.376881 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:51:28.377119 (+ 238us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:51:28.377629 (+ 510us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0924 13:51:28.377629 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0924 13:51:28.377658 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0924 13:51:28.377682 (+ 24us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 13:51:28.377834 (+ 152us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 13:51:28.377836 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 13:51:28.377838 (+ 2us) client_negotiation.cc:769] Sending connection context
0924 13:51:28.377883 (+ 45us) client_negotiation.cc:240] Negotiation successful
0924 13:51:28.377962 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":80,"threads_started":1}
I20260924 13:51:28.378661 31874 heartbeater.cc:347] Connected to a master server at 127.13.149.254:44727
I20260924 13:51:28.378751 31874 heartbeater.cc:464] Registering TS with master...
I20260924 13:51:28.378928 31874 heartbeater.cc:511] Master 127.13.149.254:44727 requested a full tablet report, sending...
I20260924 13:51:28.379294 31555 ts_manager.cc:194] Registered new tserver with Master: 60d2ac92d21b4953b0ce338030cfa553 (127.13.149.194:36153)
I20260924 13:51:28.379657 31555 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.13.149.194:37681
I20260924 13:51:28.382799 13911 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJN9WRF/build/release/bin/kudu as pid 31746
I20260924 13:51:28.382907 13911 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790257629041645-13911-0/minicluster-data/ts-1/wal/instance
I20260924 13:51:28.383072 13911 external_mini_cluster.cc:1428] Setting key 9e9520b905726785c479fe6a11bec553
I20260924 13:51:28.383961 13911 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJN9WRF/build/release/bin/kudu
/tmp/dist-test-taskJN9WRF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790257629041645-13911-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790257629041645-13911-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790257629041645-13911-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790257629041645-13911-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.149.195:0
--local_ip_for_outbound_sockets=127.13.149.195
--webserver_interface=127.13.149.195
--webserver_port=0
--tserver_master_addrs=127.13.149.254:44727
--builtin_ntp_servers=127.13.149.212:43381
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260924 13:51:28.462754 31877 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 13:51:28.462980 31877 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 13:51:28.463001 31877 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 13:51:28.464195 31877 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 13:51:28.464239 31877 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 13:51:28.464263 31877 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.149.195
I20260924 13:51:28.465694 31877 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.149.212:43381
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790257629041645-13911-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790257629041645-13911-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.149.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790257629041645-13911-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.149.195
--webserver_port=0
--tserver_master_addrs=127.13.149.254:44727
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31877
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.149.195
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790257629041645-13911-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 12:29:37 UTC on ccycloud.quasar-iqzbxn.root.comops.site
build id 4670106
I20260924 13:51:28.465942 31877 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 13:51:28.466238 31877 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 13:51:28.471153 31883 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 13:51:28.471232 31882 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 13:51:28.471154 31885 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 13:51:28.471323 31877 server_base.cc:1034] running on GCE node
I20260924 13:51:28.471521 31877 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 13:51:28.471720 31877 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 13:51:28.472847 31877 hybrid_clock.cc:648] HybridClock initialized: now 1790257888472835 us; error 28 us; skew 500 ppm
I20260924 13:51:28.474128 31877 webserver.cc:460] Webserver started at http://127.13.149.195:34363/ using document root <none> and password file <none>
I20260924 13:51:28.474371 31877 fs_manager.cc:359] Metadata directory not provided
I20260924 13:51:28.474418 31877 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 13:51:28.474546 31877 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 13:51:28.475639 31877 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790257629041645-13911-0/minicluster-data/ts-2/data/instance:
uuid: "9a6031c2caeb441b9adfb75884fc0a57"
format_stamp: "Formatted at 2026-09-24 13:51:28 on dist-test-slave-sjl2"
server_key: "50458a23dfa180ddf3f790f5b06b796a"
server_key_iv: "6a4c05524f7283d5632a06f743a3015b"
server_key_version: "encryptionkey@0"
I20260924 13:51:28.475989 31877 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790257629041645-13911-0/minicluster-data/ts-2/wal/instance:
uuid: "9a6031c2caeb441b9adfb75884fc0a57"
format_stamp: "Formatted at 2026-09-24 13:51:28 on dist-test-slave-sjl2"
server_key: "50458a23dfa180ddf3f790f5b06b796a"
server_key_iv: "6a4c05524f7283d5632a06f743a3015b"
server_key_version: "encryptionkey@0"
I20260924 13:51:28.477319 31877 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260924 13:51:28.478046 31891 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 13:51:28.478190 31877 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 13:51:28.478268 31877 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790257629041645-13911-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790257629041645-13911-0/minicluster-data/ts-2/wal
uuid: "9a6031c2caeb441b9adfb75884fc0a57"
format_stamp: "Formatted at 2026-09-24 13:51:28 on dist-test-slave-sjl2"
server_key: "50458a23dfa180ddf3f790f5b06b796a"
server_key_iv: "6a4c05524f7283d5632a06f743a3015b"
server_key_version: "encryptionkey@0"
I20260924 13:51:28.478353 31877 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790257629041645-13911-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790257629041645-13911-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790257629041645-13911-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 13:51:28.490717 31877 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 13:51:28.491010 31877 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 13:51:28.491132 31877 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 13:51:28.491339 31877 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 13:51:28.491645 31877 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 13:51:28.491678 31877 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 13:51:28.491708 31877 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 13:51:28.491731 31877 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 13:51:28.497088 31877 rpc_server.cc:307] RPC server started. Bound to: 127.13.149.195:35593
I20260924 13:51:28.497148 32004 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.149.195:35593 every 8 connection(s)
I20260924 13:51:28.497453 31877 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790257629041645-13911-0/minicluster-data/ts-2/data/info.pb
I20260924 13:51:28.498445 13911 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJN9WRF/build/release/bin/kudu as pid 31877
I20260924 13:51:28.498529 13911 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790257629041645-13911-0/minicluster-data/ts-2/wal/instance
I20260924 13:51:28.498694 13911 external_mini_cluster.cc:1428] Setting key 7a6fa009f58baaf7d9ddbadf9a415340
I20260924 13:51:28.502444 31594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:28.498133 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.149.195:40233 (local address 127.13.149.254:44727)
0924 13:51:28.498180 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0924 13:51:28.498182 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:51:28.498433 (+ 251us) server_negotiation.cc:407] Connection header received
0924 13:51:28.499048 (+ 615us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:51:28.499049 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:51:28.499062 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:51:28.499107 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:51:28.499774 (+ 667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:51:28.500258 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:51:28.501216 (+ 958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:51:28.501407 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:51:28.501461 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:51:28.502192 (+ 731us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:51:28.502193 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:51:28.502193 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 13:51:28.502198 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:51:28.502213 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 13:51:28.502217 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:51:28.502258 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0924 13:51:28.502337 (+ 79us) server_negotiation.cc:299] Negotiation successful
0924 13:51:28.502365 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260924 13:51:28.502629 32007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:28.498123 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:44727 (local address 127.13.149.195:40233)
0924 13:51:28.498315 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0924 13:51:28.498349 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0924 13:51:28.498945 (+ 596us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:51:28.499187 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:51:28.499191 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:51:28.499201 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0924 13:51:28.499687 (+ 486us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:51:28.499692 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:51:28.500343 (+ 651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:51:28.500344 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:51:28.501135 (+ 791us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:51:28.501137 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:51:28.501475 (+ 338us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:51:28.501476 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:51:28.501608 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:51:28.502067 (+ 459us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0924 13:51:28.502067 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0924 13:51:28.502094 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0924 13:51:28.502115 (+ 21us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 13:51:28.502264 (+ 149us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 13:51:28.502269 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 13:51:28.502272 (+ 3us) client_negotiation.cc:769] Sending connection context
0924 13:51:28.502335 (+ 63us) client_negotiation.cc:240] Negotiation successful
0924 13:51:28.502439 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":83,"threads_started":1}
I20260924 13:51:28.503026 32005 heartbeater.cc:347] Connected to a master server at 127.13.149.254:44727
I20260924 13:51:28.503108 32005 heartbeater.cc:464] Registering TS with master...
I20260924 13:51:28.503249 32005 heartbeater.cc:511] Master 127.13.149.254:44727 requested a full tablet report, sending...
I20260924 13:51:28.503537 31555 ts_manager.cc:194] Registered new tserver with Master: 9a6031c2caeb441b9adfb75884fc0a57 (127.13.149.195:35593)
I20260924 13:51:28.503859 31555 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.13.149.195:40233
I20260924 13:51:28.510277 13911 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260924 13:51:28.514467 31594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:28.511708 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51438 (local address 127.13.149.254:44727)
0924 13:51:28.511818 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0924 13:51:28.511820 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:51:28.511940 (+ 120us) server_negotiation.cc:407] Connection header received
0924 13:51:28.511997 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:51:28.511998 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:51:28.512008 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:51:28.512056 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:51:28.512220 (+ 164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:51:28.512700 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:51:28.513443 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:51:28.513626 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:51:28.513678 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:51:28.514244 (+ 566us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:51:28.514245 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:51:28.514245 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 13:51:28.514250 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:51:28.514264 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 13:51:28.514268 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:51:28.514317 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0924 13:51:28.514368 (+ 51us) server_negotiation.cc:299] Negotiation successful
0924 13:51:28.514391 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260924 13:51:28.515921 31555 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51438:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 13:51:28.517563 31555 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 13:51:28.523698 32013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:28.519603 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.195:35593 (local address 127.0.0.1:37158)
0924 13:51:28.519797 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0924 13:51:28.519839 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0924 13:51:28.519905 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:51:28.520411 (+ 506us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:51:28.520412 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:51:28.520414 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 13:51:28.520444 (+ 30us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:51:28.520445 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:51:28.521785 (+ 1340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:51:28.521786 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:51:28.522595 (+ 809us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:51:28.522597 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:51:28.523517 (+ 920us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:51:28.523518 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:51:28.523571 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:51:28.523574 (+ 3us) client_negotiation.cc:769] Sending connection context
0924 13:51:28.523610 (+ 36us) client_negotiation.cc:240] Negotiation successful
0924 13:51:28.523639 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":74,"threads_started":1}
I20260924 13:51:28.523756 32015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:28.519734 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37158 (local address 127.13.149.195:35593)
0924 13:51:28.520231 (+ 497us) server_negotiation.cc:206] Beginning negotiation
0924 13:51:28.520235 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 13:51:28.520245 (+ 10us) server_negotiation.cc:407] Connection header received
0924 13:51:28.520280 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:51:28.520282 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:51:28.520296 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:51:28.520415 (+ 119us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 13:51:28.520549 (+ 134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:51:28.521712 (+ 1163us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:51:28.522679 (+ 967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:51:28.523401 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:51:28.523516 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:51:28.523604 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0924 13:51:28.523630 (+ 26us) server_negotiation.cc:299] Negotiation successful
0924 13:51:28.523669 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":459,"thread_start_us":75,"threads_started":1}
I20260924 13:51:28.523909 32017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:28.519609 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.193:41703 (local address 127.0.0.1:60874)
0924 13:51:28.519966 (+ 357us) negotiation.cc:107] Waiting for socket to connect
0924 13:51:28.519976 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 13:51:28.520035 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:51:28.520208 (+ 173us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:51:28.520212 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:51:28.520214 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 13:51:28.520287 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:51:28.520288 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:51:28.521657 (+ 1369us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:51:28.521658 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:51:28.522730 (+ 1072us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:51:28.522732 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:51:28.523685 (+ 953us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:51:28.523686 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:51:28.523752 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:51:28.523756 (+ 4us) client_negotiation.cc:769] Sending connection context
0924 13:51:28.523803 (+ 47us) client_negotiation.cc:240] Negotiation successful
0924 13:51:28.523843 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":338,"mutex_wait_us":7,"thread_start_us":83,"threads_started":1}
I20260924 13:51:28.523698 32014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:28.519821 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.194:36153 (local address 127.0.0.1:44448)
0924 13:51:28.520002 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0924 13:51:28.520010 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0924 13:51:28.520045 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:51:28.520310 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:51:28.520311 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:51:28.520313 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 13:51:28.520365 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:51:28.520367 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:51:28.521449 (+ 1082us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:51:28.521450 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:51:28.522694 (+ 1244us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:51:28.522697 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:51:28.523382 (+ 685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:51:28.523384 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:51:28.523463 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:51:28.523475 (+ 12us) client_negotiation.cc:769] Sending connection context
0924 13:51:28.523539 (+ 64us) client_negotiation.cc:240] Negotiation successful
0924 13:51:28.523595 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":69,"threads_started":1}
I20260924 13:51:28.524151 32016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:28.519776 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60874 (local address 127.13.149.193:41703)
0924 13:51:28.519970 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0924 13:51:28.519975 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 13:51:28.520028 (+ 53us) server_negotiation.cc:407] Connection header received
0924 13:51:28.520102 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:51:28.520104 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:51:28.520119 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:51:28.520192 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 13:51:28.520378 (+ 186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:51:28.521542 (+ 1164us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:51:28.522798 (+ 1256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:51:28.523517 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:51:28.523573 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:51:28.523652 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0924 13:51:28.524048 (+ 396us) server_negotiation.cc:299] Negotiation successful
0924 13:51:28.524090 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":161,"thread_start_us":86,"threads_started":1}
I20260924 13:51:28.524359 32018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:28.520031 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44448 (local address 127.13.149.194:36153)
0924 13:51:28.520186 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0924 13:51:28.520190 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 13:51:28.520199 (+ 9us) server_negotiation.cc:407] Connection header received
0924 13:51:28.520232 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:51:28.520233 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:51:28.520247 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:51:28.520311 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 13:51:28.520515 (+ 204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:51:28.521375 (+ 860us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:51:28.522796 (+ 1421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:51:28.523307 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:51:28.523373 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:51:28.523468 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0924 13:51:28.523743 (+ 275us) server_negotiation.cc:299] Negotiation successful
0924 13:51:28.524278 (+ 535us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":117,"thread_start_us":79,"threads_started":1}
I20260924 13:51:28.524487 31939 tablet_service.cc:1467] Processing CreateTablet for tablet c27706e32f4e414fb78070ff1c7bfcb5 (DEFAULT_TABLE table=test-table [id=76b23d21603b455aaf47ac7696fcfae5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 13:51:28.524803 31939 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c27706e32f4e414fb78070ff1c7bfcb5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 13:51:28.524996 31808 tablet_service.cc:1467] Processing CreateTablet for tablet c27706e32f4e414fb78070ff1c7bfcb5 (DEFAULT_TABLE table=test-table [id=76b23d21603b455aaf47ac7696fcfae5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 13:51:28.525336 31808 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c27706e32f4e414fb78070ff1c7bfcb5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 13:51:28.527858 32020 tablet_bootstrap.cc:492] T c27706e32f4e414fb78070ff1c7bfcb5 P 9a6031c2caeb441b9adfb75884fc0a57: Bootstrap starting.
I20260924 13:51:28.527942 32019 tablet_bootstrap.cc:492] T c27706e32f4e414fb78070ff1c7bfcb5 P 60d2ac92d21b4953b0ce338030cfa553: Bootstrap starting.
I20260924 13:51:28.528328 31677 tablet_service.cc:1467] Processing CreateTablet for tablet c27706e32f4e414fb78070ff1c7bfcb5 (DEFAULT_TABLE table=test-table [id=76b23d21603b455aaf47ac7696fcfae5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 13:51:28.528458 32020 tablet_bootstrap.cc:654] T c27706e32f4e414fb78070ff1c7bfcb5 P 9a6031c2caeb441b9adfb75884fc0a57: Neither blocks nor log segments found. Creating new log.
I20260924 13:51:28.528579 32019 tablet_bootstrap.cc:654] T c27706e32f4e414fb78070ff1c7bfcb5 P 60d2ac92d21b4953b0ce338030cfa553: Neither blocks nor log segments found. Creating new log.
I20260924 13:51:28.528667 31677 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c27706e32f4e414fb78070ff1c7bfcb5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 13:51:28.528772 32020 log.cc:824] T c27706e32f4e414fb78070ff1c7bfcb5 P 9a6031c2caeb441b9adfb75884fc0a57: Log is configured to *not* fsync() on all Append() calls
I20260924 13:51:28.528898 32019 log.cc:824] T c27706e32f4e414fb78070ff1c7bfcb5 P 60d2ac92d21b4953b0ce338030cfa553: Log is configured to *not* fsync() on all Append() calls
I20260924 13:51:28.529646 32019 tablet_bootstrap.cc:492] T c27706e32f4e414fb78070ff1c7bfcb5 P 60d2ac92d21b4953b0ce338030cfa553: No bootstrap required, opened a new log
I20260924 13:51:28.529717 32019 ts_tablet_manager.cc:1397] T c27706e32f4e414fb78070ff1c7bfcb5 P 60d2ac92d21b4953b0ce338030cfa553: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260924 13:51:28.531910 32023 tablet_bootstrap.cc:492] T c27706e32f4e414fb78070ff1c7bfcb5 P 14188b1e6ba344399d4ed134b6c06e1a: Bootstrap starting.
I20260924 13:51:28.532191 32019 raft_consensus.cc:348] T c27706e32f4e414fb78070ff1c7bfcb5 P 60d2ac92d21b4953b0ce338030cfa553 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14188b1e6ba344399d4ed134b6c06e1a" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 41703 } } peers { permanent_uuid: "9a6031c2caeb441b9adfb75884fc0a57" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 35593 } } peers { permanent_uuid: "60d2ac92d21b4953b0ce338030cfa553" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 36153 } }
I20260924 13:51:28.532349 32019 raft_consensus.cc:374] T c27706e32f4e414fb78070ff1c7bfcb5 P 60d2ac92d21b4953b0ce338030cfa553 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 13:51:28.532380 32019 raft_consensus.cc:729] T c27706e32f4e414fb78070ff1c7bfcb5 P 60d2ac92d21b4953b0ce338030cfa553 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60d2ac92d21b4953b0ce338030cfa553, State: Initialized, Role: FOLLOWER
I20260924 13:51:28.532546 32023 tablet_bootstrap.cc:654] T c27706e32f4e414fb78070ff1c7bfcb5 P 14188b1e6ba344399d4ed134b6c06e1a: Neither blocks nor log segments found. Creating new log.
I20260924 13:51:28.532518 32019 consensus_queue.cc:260] T c27706e32f4e414fb78070ff1c7bfcb5 P 60d2ac92d21b4953b0ce338030cfa553 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14188b1e6ba344399d4ed134b6c06e1a" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 41703 } } peers { permanent_uuid: "9a6031c2caeb441b9adfb75884fc0a57" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 35593 } } peers { permanent_uuid: "60d2ac92d21b4953b0ce338030cfa553" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 36153 } }
I20260924 13:51:28.532572 32020 tablet_bootstrap.cc:492] T c27706e32f4e414fb78070ff1c7bfcb5 P 9a6031c2caeb441b9adfb75884fc0a57: No bootstrap required, opened a new log
I20260924 13:51:28.532630 32020 ts_tablet_manager.cc:1397] T c27706e32f4e414fb78070ff1c7bfcb5 P 9a6031c2caeb441b9adfb75884fc0a57: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.001s
I20260924 13:51:28.532768 32019 ts_tablet_manager.cc:1428] T c27706e32f4e414fb78070ff1c7bfcb5 P 60d2ac92d21b4953b0ce338030cfa553: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260924 13:51:28.532904 31874 heartbeater.cc:503] Master 127.13.149.254:44727 was elected leader, sending a full tablet report...
I20260924 13:51:28.532891 32023 log.cc:824] T c27706e32f4e414fb78070ff1c7bfcb5 P 14188b1e6ba344399d4ed134b6c06e1a: Log is configured to *not* fsync() on all Append() calls
I20260924 13:51:28.533667 32023 tablet_bootstrap.cc:492] T c27706e32f4e414fb78070ff1c7bfcb5 P 14188b1e6ba344399d4ed134b6c06e1a: No bootstrap required, opened a new log
I20260924 13:51:28.533731 32023 ts_tablet_manager.cc:1397] T c27706e32f4e414fb78070ff1c7bfcb5 P 14188b1e6ba344399d4ed134b6c06e1a: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260924 13:51:28.535225 32020 raft_consensus.cc:348] T c27706e32f4e414fb78070ff1c7bfcb5 P 9a6031c2caeb441b9adfb75884fc0a57 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14188b1e6ba344399d4ed134b6c06e1a" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 41703 } } peers { permanent_uuid: "9a6031c2caeb441b9adfb75884fc0a57" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 35593 } } peers { permanent_uuid: "60d2ac92d21b4953b0ce338030cfa553" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 36153 } }
I20260924 13:51:28.535413 32020 raft_consensus.cc:374] T c27706e32f4e414fb78070ff1c7bfcb5 P 9a6031c2caeb441b9adfb75884fc0a57 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 13:51:28.535458 32020 raft_consensus.cc:729] T c27706e32f4e414fb78070ff1c7bfcb5 P 9a6031c2caeb441b9adfb75884fc0a57 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9a6031c2caeb441b9adfb75884fc0a57, State: Initialized, Role: FOLLOWER
I20260924 13:51:28.535591 32020 consensus_queue.cc:260] T c27706e32f4e414fb78070ff1c7bfcb5 P 9a6031c2caeb441b9adfb75884fc0a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14188b1e6ba344399d4ed134b6c06e1a" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 41703 } } peers { permanent_uuid: "9a6031c2caeb441b9adfb75884fc0a57" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 35593 } } peers { permanent_uuid: "60d2ac92d21b4953b0ce338030cfa553" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 36153 } }
I20260924 13:51:28.535849 32020 ts_tablet_manager.cc:1428] T c27706e32f4e414fb78070ff1c7bfcb5 P 9a6031c2caeb441b9adfb75884fc0a57: Time spent starting tablet: real 0.003s user 0.000s sys 0.004s
I20260924 13:51:28.535935 32005 heartbeater.cc:503] Master 127.13.149.254:44727 was elected leader, sending a full tablet report...
I20260924 13:51:28.536398 32023 raft_consensus.cc:348] T c27706e32f4e414fb78070ff1c7bfcb5 P 14188b1e6ba344399d4ed134b6c06e1a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14188b1e6ba344399d4ed134b6c06e1a" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 41703 } } peers { permanent_uuid: "9a6031c2caeb441b9adfb75884fc0a57" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 35593 } } peers { permanent_uuid: "60d2ac92d21b4953b0ce338030cfa553" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 36153 } }
I20260924 13:51:28.536546 32023 raft_consensus.cc:374] T c27706e32f4e414fb78070ff1c7bfcb5 P 14188b1e6ba344399d4ed134b6c06e1a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 13:51:28.536573 32023 raft_consensus.cc:729] T c27706e32f4e414fb78070ff1c7bfcb5 P 14188b1e6ba344399d4ed134b6c06e1a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 14188b1e6ba344399d4ed134b6c06e1a, State: Initialized, Role: FOLLOWER
I20260924 13:51:28.536669 32023 consensus_queue.cc:260] T c27706e32f4e414fb78070ff1c7bfcb5 P 14188b1e6ba344399d4ed134b6c06e1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14188b1e6ba344399d4ed134b6c06e1a" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 41703 } } peers { permanent_uuid: "9a6031c2caeb441b9adfb75884fc0a57" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 35593 } } peers { permanent_uuid: "60d2ac92d21b4953b0ce338030cfa553" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 36153 } }
I20260924 13:51:28.536962 32023 ts_tablet_manager.cc:1428] T c27706e32f4e414fb78070ff1c7bfcb5 P 14188b1e6ba344399d4ed134b6c06e1a: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260924 13:51:28.536989 31743 heartbeater.cc:503] Master 127.13.149.254:44727 was elected leader, sending a full tablet report...
I20260924 13:51:28.579885 32026 raft_consensus.cc:482] T c27706e32f4e414fb78070ff1c7bfcb5 P 9a6031c2caeb441b9adfb75884fc0a57 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 13:51:28.580022 32026 raft_consensus.cc:504] T c27706e32f4e414fb78070ff1c7bfcb5 P 9a6031c2caeb441b9adfb75884fc0a57 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14188b1e6ba344399d4ed134b6c06e1a" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 41703 } } peers { permanent_uuid: "9a6031c2caeb441b9adfb75884fc0a57" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 35593 } } peers { permanent_uuid: "60d2ac92d21b4953b0ce338030cfa553" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 36153 } }
I20260924 13:51:28.580369 32026 leader_election.cc:288] T c27706e32f4e414fb78070ff1c7bfcb5 P 9a6031c2caeb441b9adfb75884fc0a57 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 14188b1e6ba344399d4ed134b6c06e1a (127.13.149.193:41703), 60d2ac92d21b4953b0ce338030cfa553 (127.13.149.194:36153)
I20260924 13:51:28.583441 32018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:28.580677 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.149.195:58599 (local address 127.13.149.194:36153)
0924 13:51:28.580733 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0924 13:51:28.580735 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:51:28.580853 (+ 118us) server_negotiation.cc:407] Connection header received
0924 13:51:28.580937 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:51:28.580938 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:51:28.580947 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:51:28.580994 (+ 47us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 13:51:28.581118 (+ 124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:51:28.581786 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:51:28.582743 (+ 957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:51:28.583124 (+ 381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:51:28.583184 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:51:28.583193 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0924 13:51:28.583331 (+ 138us) server_negotiation.cc:299] Negotiation successful
0924 13:51:28.583367 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260924 13:51:28.583458 32016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:28.580589 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.149.195:46017 (local address 127.13.149.193:41703)
0924 13:51:28.580661 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0924 13:51:28.580663 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:51:28.580674 (+ 11us) server_negotiation.cc:407] Connection header received
0924 13:51:28.580703 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:51:28.580703 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:51:28.580715 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:51:28.580785 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 13:51:28.580973 (+ 188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:51:28.581711 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:51:28.582768 (+ 1057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:51:28.583143 (+ 375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:51:28.583206 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:51:28.583215 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0924 13:51:28.583351 (+ 136us) server_negotiation.cc:299] Negotiation successful
0924 13:51:28.583393 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260924 13:51:28.583458 32007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:28.580510 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.193:41703 (local address 127.13.149.195:46017)
0924 13:51:28.580581 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0924 13:51:28.580593 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0924 13:51:28.580638 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:51:28.580813 (+ 175us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:51:28.580814 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:51:28.580816 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 13:51:28.580891 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:51:28.580893 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:51:28.581797 (+ 904us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:51:28.581797 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:51:28.582677 (+ 880us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:51:28.582679 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:51:28.583214 (+ 535us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:51:28.583214 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:51:28.583286 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:51:28.583289 (+ 3us) client_negotiation.cc:769] Sending connection context
0924 13:51:28.583335 (+ 46us) client_negotiation.cc:240] Negotiation successful
0924 13:51:28.583378 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":48,"mutex_wait_us":42}
I20260924 13:51:28.583477 32028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:28.580623 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.194:36153 (local address 127.13.149.195:58599)
0924 13:51:28.580780 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0924 13:51:28.580809 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0924 13:51:28.580859 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:51:28.581010 (+ 151us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:51:28.581010 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:51:28.581012 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 13:51:28.581063 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:51:28.581064 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:51:28.581861 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:51:28.581862 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:51:28.582657 (+ 795us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:51:28.582660 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:51:28.583199 (+ 539us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:51:28.583200 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:51:28.583272 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:51:28.583275 (+ 3us) client_negotiation.cc:769] Sending connection context
0924 13:51:28.583326 (+ 51us) client_negotiation.cc:240] Negotiation successful
0924 13:51:28.583367 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"mutex_wait_us":3,"thread_start_us":104,"threads_started":1}
I20260924 13:51:28.583782 31697 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c27706e32f4e414fb78070ff1c7bfcb5" candidate_uuid: "9a6031c2caeb441b9adfb75884fc0a57" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "14188b1e6ba344399d4ed134b6c06e1a" is_pre_election: true
I20260924 13:51:28.583907 31828 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c27706e32f4e414fb78070ff1c7bfcb5" candidate_uuid: "9a6031c2caeb441b9adfb75884fc0a57" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60d2ac92d21b4953b0ce338030cfa553" is_pre_election: true
I20260924 13:51:28.583967 31697 raft_consensus.cc:2454] T c27706e32f4e414fb78070ff1c7bfcb5 P 14188b1e6ba344399d4ed134b6c06e1a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9a6031c2caeb441b9adfb75884fc0a57 in term 0.
I20260924 13:51:28.584033 31828 raft_consensus.cc:2454] T c27706e32f4e414fb78070ff1c7bfcb5 P 60d2ac92d21b4953b0ce338030cfa553 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9a6031c2caeb441b9adfb75884fc0a57 in term 0.
I20260924 13:51:28.584182 31892 leader_election.cc:302] T c27706e32f4e414fb78070ff1c7bfcb5 P 9a6031c2caeb441b9adfb75884fc0a57 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 14188b1e6ba344399d4ed134b6c06e1a, 9a6031c2caeb441b9adfb75884fc0a57; no voters:
I20260924 13:51:28.584307 32026 raft_consensus.cc:2781] T c27706e32f4e414fb78070ff1c7bfcb5 P 9a6031c2caeb441b9adfb75884fc0a57 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 13:51:28.584367 32026 raft_consensus.cc:482] T c27706e32f4e414fb78070ff1c7bfcb5 P 9a6031c2caeb441b9adfb75884fc0a57 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 13:51:28.584390 32026 raft_consensus.cc:3037] T c27706e32f4e414fb78070ff1c7bfcb5 P 9a6031c2caeb441b9adfb75884fc0a57 [term 0 FOLLOWER]: Advancing to term 1
I20260924 13:51:28.585112 32026 raft_consensus.cc:504] T c27706e32f4e414fb78070ff1c7bfcb5 P 9a6031c2caeb441b9adfb75884fc0a57 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14188b1e6ba344399d4ed134b6c06e1a" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 41703 } } peers { permanent_uuid: "9a6031c2caeb441b9adfb75884fc0a57" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 35593 } } peers { permanent_uuid: "60d2ac92d21b4953b0ce338030cfa553" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 36153 } }
I20260924 13:51:28.585232 32026 leader_election.cc:288] T c27706e32f4e414fb78070ff1c7bfcb5 P 9a6031c2caeb441b9adfb75884fc0a57 [CANDIDATE]: Term 1 election: Requested vote from peers 14188b1e6ba344399d4ed134b6c06e1a (127.13.149.193:41703), 60d2ac92d21b4953b0ce338030cfa553 (127.13.149.194:36153)
I20260924 13:51:28.585405 31828 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c27706e32f4e414fb78070ff1c7bfcb5" candidate_uuid: "9a6031c2caeb441b9adfb75884fc0a57" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60d2ac92d21b4953b0ce338030cfa553"
I20260924 13:51:28.585484 31828 raft_consensus.cc:3037] T c27706e32f4e414fb78070ff1c7bfcb5 P 60d2ac92d21b4953b0ce338030cfa553 [term 0 FOLLOWER]: Advancing to term 1
I20260924 13:51:28.586189 31828 raft_consensus.cc:2454] T c27706e32f4e414fb78070ff1c7bfcb5 P 60d2ac92d21b4953b0ce338030cfa553 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9a6031c2caeb441b9adfb75884fc0a57 in term 1.
I20260924 13:51:28.586336 31697 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c27706e32f4e414fb78070ff1c7bfcb5" candidate_uuid: "9a6031c2caeb441b9adfb75884fc0a57" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "14188b1e6ba344399d4ed134b6c06e1a"
I20260924 13:51:28.586371 31892 leader_election.cc:302] T c27706e32f4e414fb78070ff1c7bfcb5 P 9a6031c2caeb441b9adfb75884fc0a57 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 60d2ac92d21b4953b0ce338030cfa553, 9a6031c2caeb441b9adfb75884fc0a57; no voters:
I20260924 13:51:28.586408 31697 raft_consensus.cc:3037] T c27706e32f4e414fb78070ff1c7bfcb5 P 14188b1e6ba344399d4ed134b6c06e1a [term 0 FOLLOWER]: Advancing to term 1
I20260924 13:51:28.586462 32026 raft_consensus.cc:2781] T c27706e32f4e414fb78070ff1c7bfcb5 P 9a6031c2caeb441b9adfb75884fc0a57 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 13:51:28.586627 32026 raft_consensus.cc:686] T c27706e32f4e414fb78070ff1c7bfcb5 P 9a6031c2caeb441b9adfb75884fc0a57 [term 1 LEADER]: Becoming Leader. State: Replica: 9a6031c2caeb441b9adfb75884fc0a57, State: Running, Role: LEADER
I20260924 13:51:28.586726 32026 consensus_queue.cc:237] T c27706e32f4e414fb78070ff1c7bfcb5 P 9a6031c2caeb441b9adfb75884fc0a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14188b1e6ba344399d4ed134b6c06e1a" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 41703 } } peers { permanent_uuid: "9a6031c2caeb441b9adfb75884fc0a57" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 35593 } } peers { permanent_uuid: "60d2ac92d21b4953b0ce338030cfa553" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 36153 } }
I20260924 13:51:28.587205 31697 raft_consensus.cc:2454] T c27706e32f4e414fb78070ff1c7bfcb5 P 14188b1e6ba344399d4ed134b6c06e1a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9a6031c2caeb441b9adfb75884fc0a57 in term 1.
I20260924 13:51:28.587592 31555 catalog_manager.cc:5495] T c27706e32f4e414fb78070ff1c7bfcb5 P 9a6031c2caeb441b9adfb75884fc0a57 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9a6031c2caeb441b9adfb75884fc0a57 (127.13.149.195). New cstate: current_term: 1 leader_uuid: "9a6031c2caeb441b9adfb75884fc0a57" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14188b1e6ba344399d4ed134b6c06e1a" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 41703 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9a6031c2caeb441b9adfb75884fc0a57" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 35593 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "60d2ac92d21b4953b0ce338030cfa553" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 36153 } health_report { overall_health: UNKNOWN } } }
I20260924 13:51:28.596665 32015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:28.594433 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37166 (local address 127.13.149.195:35593)
0924 13:51:28.594493 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0924 13:51:28.594496 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 13:51:28.594504 (+ 8us) server_negotiation.cc:407] Connection header received
0924 13:51:28.594548 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:51:28.594549 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:51:28.594570 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:51:28.594628 (+ 58us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 13:51:28.594740 (+ 112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:51:28.595207 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:51:28.595942 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:51:28.596133 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:51:28.596187 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:51:28.596335 (+ 148us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 13:51:28.596437 (+ 102us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 13:51:28.596507 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0924 13:51:28.596545 (+ 38us) server_negotiation.cc:299] Negotiation successful
0924 13:51:28.596564 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260924 13:51:28.598464 31828 raft_consensus.cc:1261] T c27706e32f4e414fb78070ff1c7bfcb5 P 60d2ac92d21b4953b0ce338030cfa553 [term 1 FOLLOWER]: Refusing update from remote peer 9a6031c2caeb441b9adfb75884fc0a57: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 13:51:28.598465 31697 raft_consensus.cc:1261] T c27706e32f4e414fb78070ff1c7bfcb5 P 14188b1e6ba344399d4ed134b6c06e1a [term 1 FOLLOWER]: Refusing update from remote peer 9a6031c2caeb441b9adfb75884fc0a57: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 13:51:28.598806 32029 consensus_queue.cc:1035] T c27706e32f4e414fb78070ff1c7bfcb5 P 9a6031c2caeb441b9adfb75884fc0a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "14188b1e6ba344399d4ed134b6c06e1a" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 41703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 13:51:28.598915 32026 consensus_queue.cc:1035] T c27706e32f4e414fb78070ff1c7bfcb5 P 9a6031c2caeb441b9adfb75884fc0a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "60d2ac92d21b4953b0ce338030cfa553" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 36153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 13:51:28.603492 31555 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:51438:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 13:51:28.603593 31555 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:51438:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 13:51:28.604321 31555 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3c21785cfa5c42e28cbc40ea271097c5: Sending DeleteTablet for 3 replicas of tablet c27706e32f4e414fb78070ff1c7bfcb5
I20260924 13:51:28.604619 31677 tablet_service.cc:1514] Processing DeleteTablet for tablet c27706e32f4e414fb78070ff1c7bfcb5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 13:51:28 UTC) from {username='slave'} at 127.0.0.1:60874
I20260924 13:51:28.604650 31808 tablet_service.cc:1514] Processing DeleteTablet for tablet c27706e32f4e414fb78070ff1c7bfcb5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 13:51:28 UTC) from {username='slave'} at 127.0.0.1:44448
I20260924 13:51:28.604880 31939 tablet_service.cc:1514] Processing DeleteTablet for tablet c27706e32f4e414fb78070ff1c7bfcb5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 13:51:28 UTC) from {username='slave'} at 127.0.0.1:37158
I20260924 13:51:28.604962 13911 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJN9WRF/build/release/bin/kudu with pid 31615
I20260924 13:51:28.604889 32043 tablet_replica.cc:330] T c27706e32f4e414fb78070ff1c7bfcb5 P 60d2ac92d21b4953b0ce338030cfa553: stopping tablet replica
I20260924 13:51:28.608412 32043 raft_consensus.cc:2229] T c27706e32f4e414fb78070ff1c7bfcb5 P 60d2ac92d21b4953b0ce338030cfa553 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 13:51:28.608542 32043 raft_consensus.cc:2258] T c27706e32f4e414fb78070ff1c7bfcb5 P 60d2ac92d21b4953b0ce338030cfa553 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 13:51:28.609014 32044 tablet_replica.cc:330] T c27706e32f4e414fb78070ff1c7bfcb5 P 9a6031c2caeb441b9adfb75884fc0a57: stopping tablet replica
I20260924 13:51:28.609197 32044 raft_consensus.cc:2229] T c27706e32f4e414fb78070ff1c7bfcb5 P 9a6031c2caeb441b9adfb75884fc0a57 [term 1 LEADER]: Raft consensus shutting down.
I20260924 13:51:28.609333 32044 raft_consensus.cc:2258] T c27706e32f4e414fb78070ff1c7bfcb5 P 9a6031c2caeb441b9adfb75884fc0a57 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 13:51:28.609747 32044 ts_tablet_manager.cc:1905] T c27706e32f4e414fb78070ff1c7bfcb5 P 9a6031c2caeb441b9adfb75884fc0a57: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 13:51:28.611387 32044 ts_tablet_manager.cc:1918] T c27706e32f4e414fb78070ff1c7bfcb5 P 9a6031c2caeb441b9adfb75884fc0a57: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 13:51:28.611451 32044 log.cc:1196] T c27706e32f4e414fb78070ff1c7bfcb5 P 9a6031c2caeb441b9adfb75884fc0a57: Deleting WAL directory at /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790257629041645-13911-0/minicluster-data/ts-2/wal/wals/c27706e32f4e414fb78070ff1c7bfcb5
I20260924 13:51:28.611670 32044 ts_tablet_manager.cc:1939] T c27706e32f4e414fb78070ff1c7bfcb5 P 9a6031c2caeb441b9adfb75884fc0a57: Deleting consensus metadata
I20260924 13:51:28.612102 31540 catalog_manager.cc:4841] TS 9a6031c2caeb441b9adfb75884fc0a57 (127.13.149.195:35593): tablet c27706e32f4e414fb78070ff1c7bfcb5 (table test-table [id=76b23d21603b455aaf47ac7696fcfae5]) successfully deleted
W20260924 13:51:28.612505 31539 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.13.149.193:41703 (error 108)
I20260924 13:51:28.612982 32017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:28.612842 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.193:41703 (local address 127.0.0.1:60884)
0924 13:51:28.612888 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0924 13:51:28.612949 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.149.193:41703: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":21}
I20260924 13:51:28.613127 32043 ts_tablet_manager.cc:1905] T c27706e32f4e414fb78070ff1c7bfcb5 P 60d2ac92d21b4953b0ce338030cfa553: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260924 13:51:28.613129 31539 catalog_manager.cc:4584] TS 14188b1e6ba344399d4ed134b6c06e1a (127.13.149.193:41703): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet c27706e32f4e414fb78070ff1c7bfcb5: Network error: Client connection negotiation failed: client connection to 127.13.149.193:41703: connect: Connection refused (error 111)
I20260924 13:51:28.613286 13911 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJN9WRF/build/release/bin/kudu with pid 31746
I20260924 13:51:28.617272 32017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:28.617144 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.194:36153 (local address 127.0.0.1:44450)
0924 13:51:28.617199 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0924 13:51:28.617234 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.149.194:36153: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
I20260924 13:51:28.617831 13911 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJN9WRF/build/release/bin/kudu with pid 31877
I20260924 13:51:28.621817 13911 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJN9WRF/build/release/bin/kudu with pid 31524
2026-09-24T13:51:28Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (670 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790257629041645-13911-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 13:51:28 dist-test-slave-sjl2 krb5kdc[32048](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 13:51:28 dist-test-slave-sjl2 krb5kdc[32048](info): set up 2 sockets
Sep 24 13:51:28 dist-test-slave-sjl2 krb5kdc[32048](info): commencing operation
krb5kdc: starting...
W20260924 13:51:29.569536 13911 mini_kdc.cc:121] Time spent starting KDC: real 0.929s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790257629041645-13911-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790257629041645-13911-0/krb5kdc/test-user.keytab.
Sep 24 13:51:29 dist-test-slave-sjl2 krb5kdc[32048](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257889, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T13:51:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T13:51:29Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.149.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.149.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.149.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.254.keytab.
Entry for principal kudu/127.13.149.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.254.keytab.
I20260924 13:51:29.684481 13911 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJN9WRF/build/release/bin/kudu
/tmp/dist-test-taskJN9WRF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790257629041645-13911-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790257629041645-13911-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790257629041645-13911-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790257629041645-13911-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.149.254:43725
--webserver_interface=127.13.149.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.254.keytab
--principal=kudu/127.13.149.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.149.212:41733
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.149.254:43725
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790257629041645-13911-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790257629041645-13911-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790257629041645-13911-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 13:51:29.766239 32061 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 13:51:29.766458 32061 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 13:51:29.766482 32061 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 13:51:29.767611 32061 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 13:51:29.767644 32061 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 13:51:29.767658 32061 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 13:51:29.767668 32061 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 13:51:29.767679 32061 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 13:51:29.769147 32061 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.149.212:41733
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790257629041645-13911-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790257629041645-13911-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.149.254:43725
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.254.keytab
--principal=kudu/127.13.149.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.149.254:43725
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790257629041645-13911-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.149.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32061
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790257629041645-13911-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 12:29:37 UTC on ccycloud.quasar-iqzbxn.root.comops.site
build id 4670106
I20260924 13:51:29.769344 32061 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 13:51:29.769518 32061 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260924 13:51:29.774360 32061 server_base.cc:1034] running on GCE node
W20260924 13:51:29.774343 32067 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 13:51:29.774343 32069 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 13:51:29.774375 32066 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 13:51:29.774750 32061 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 13:51:29.775013 32061 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 13:51:29.776151 32061 hybrid_clock.cc:648] HybridClock initialized: now 1790257889776154 us; error 59 us; skew 500 ppm
Sep 24 13:51:29 dist-test-slave-sjl2 krb5kdc[32048](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257889, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.149.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 13:51:29.778618 32061 init.cc:377] Logged in from keytab as kudu/127.13.149.254@KRBTEST.COM (short username kudu)
I20260924 13:51:29.779063 32061 webserver.cc:460] Webserver started at http://127.13.149.254:39395/ using document root <none> and password file <none>
I20260924 13:51:29.779243 32061 fs_manager.cc:359] Metadata directory not provided
I20260924 13:51:29.779281 32061 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 13:51:29.779385 32061 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 13:51:29.780426 32061 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790257629041645-13911-0/minicluster-data/master-0/data/instance:
uuid: "48551c20c2a549bab8866b40f9ff3dab"
format_stamp: "Formatted at 2026-09-24 13:51:29 on dist-test-slave-sjl2"
server_key: "0ca182067c32168de420c6d964544ba5"
server_key_iv: "ef88aacc00a02be38fdc07d347c17bbf"
server_key_version: "encryptionkey@0"
I20260924 13:51:29.780733 32061 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790257629041645-13911-0/minicluster-data/master-0/wal/instance:
uuid: "48551c20c2a549bab8866b40f9ff3dab"
format_stamp: "Formatted at 2026-09-24 13:51:29 on dist-test-slave-sjl2"
server_key: "0ca182067c32168de420c6d964544ba5"
server_key_iv: "ef88aacc00a02be38fdc07d347c17bbf"
server_key_version: "encryptionkey@0"
I20260924 13:51:29.781993 32061 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260924 13:51:29.782671 32076 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 13:51:29.782823 32061 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 13:51:29.782907 32061 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790257629041645-13911-0/minicluster-data/master-0/data,/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790257629041645-13911-0/minicluster-data/master-0/wal
uuid: "48551c20c2a549bab8866b40f9ff3dab"
format_stamp: "Formatted at 2026-09-24 13:51:29 on dist-test-slave-sjl2"
server_key: "0ca182067c32168de420c6d964544ba5"
server_key_iv: "ef88aacc00a02be38fdc07d347c17bbf"
server_key_version: "encryptionkey@0"
I20260924 13:51:29.782969 32061 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790257629041645-13911-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790257629041645-13911-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790257629041645-13911-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 13:51:29.794348 32061 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 13:51:29.796191 32061 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 13:51:29.796336 32061 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 13:51:29.800110 32061 rpc_server.cc:307] RPC server started. Bound to: 127.13.149.254:43725
I20260924 13:51:29.800155 32128 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.149.254:43725 every 8 connection(s)
I20260924 13:51:29.800473 32061 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790257629041645-13911-0/minicluster-data/master-0/data/info.pb
I20260924 13:51:29.801329 32129 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 13:51:29.803532 32129 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 48551c20c2a549bab8866b40f9ff3dab: Bootstrap starting.
I20260924 13:51:29.804068 32129 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 48551c20c2a549bab8866b40f9ff3dab: Neither blocks nor log segments found. Creating new log.
I20260924 13:51:29.804323 32129 log.cc:824] T 00000000000000000000000000000000 P 48551c20c2a549bab8866b40f9ff3dab: Log is configured to *not* fsync() on all Append() calls
I20260924 13:51:29.805024 32129 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 48551c20c2a549bab8866b40f9ff3dab: No bootstrap required, opened a new log
I20260924 13:51:29.806636 32129 raft_consensus.cc:348] T 00000000000000000000000000000000 P 48551c20c2a549bab8866b40f9ff3dab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48551c20c2a549bab8866b40f9ff3dab" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 43725 } }
I20260924 13:51:29.806779 32129 raft_consensus.cc:374] T 00000000000000000000000000000000 P 48551c20c2a549bab8866b40f9ff3dab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 13:51:29.806802 32129 raft_consensus.cc:729] T 00000000000000000000000000000000 P 48551c20c2a549bab8866b40f9ff3dab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48551c20c2a549bab8866b40f9ff3dab, State: Initialized, Role: FOLLOWER
I20260924 13:51:29.806943 32129 consensus_queue.cc:260] T 00000000000000000000000000000000 P 48551c20c2a549bab8866b40f9ff3dab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48551c20c2a549bab8866b40f9ff3dab" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 43725 } }
I20260924 13:51:29.807009 32129 raft_consensus.cc:388] T 00000000000000000000000000000000 P 48551c20c2a549bab8866b40f9ff3dab [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 13:51:29.807031 32129 raft_consensus.cc:482] T 00000000000000000000000000000000 P 48551c20c2a549bab8866b40f9ff3dab [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 13:51:29.807078 32129 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 48551c20c2a549bab8866b40f9ff3dab [term 0 FOLLOWER]: Advancing to term 1
I20260924 13:51:29.807626 32129 raft_consensus.cc:504] T 00000000000000000000000000000000 P 48551c20c2a549bab8866b40f9ff3dab [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48551c20c2a549bab8866b40f9ff3dab" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 43725 } }
I20260924 13:51:29.807715 32129 leader_election.cc:302] T 00000000000000000000000000000000 P 48551c20c2a549bab8866b40f9ff3dab [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 48551c20c2a549bab8866b40f9ff3dab; no voters:
I20260924 13:51:29.807864 32129 leader_election.cc:288] T 00000000000000000000000000000000 P 48551c20c2a549bab8866b40f9ff3dab [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 13:51:29.807914 32132 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 48551c20c2a549bab8866b40f9ff3dab [term 1 FOLLOWER]: Leader election won for term 1
I20260924 13:51:29.808109 32129 sys_catalog.cc:564] T 00000000000000000000000000000000 P 48551c20c2a549bab8866b40f9ff3dab [sys.catalog]: configured and running, proceeding with master startup.
I20260924 13:51:29.808125 32132 raft_consensus.cc:686] T 00000000000000000000000000000000 P 48551c20c2a549bab8866b40f9ff3dab [term 1 LEADER]: Becoming Leader. State: Replica: 48551c20c2a549bab8866b40f9ff3dab, State: Running, Role: LEADER
I20260924 13:51:29.808249 32132 consensus_queue.cc:237] T 00000000000000000000000000000000 P 48551c20c2a549bab8866b40f9ff3dab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48551c20c2a549bab8866b40f9ff3dab" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 43725 } }
I20260924 13:51:29.808669 32133 sys_catalog.cc:455] T 00000000000000000000000000000000 P 48551c20c2a549bab8866b40f9ff3dab [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "48551c20c2a549bab8866b40f9ff3dab" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48551c20c2a549bab8866b40f9ff3dab" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 43725 } } }
I20260924 13:51:29.808755 32133 sys_catalog.cc:458] T 00000000000000000000000000000000 P 48551c20c2a549bab8866b40f9ff3dab [sys.catalog]: This master's current role is: LEADER
I20260924 13:51:29.808802 32134 sys_catalog.cc:455] T 00000000000000000000000000000000 P 48551c20c2a549bab8866b40f9ff3dab [sys.catalog]: SysCatalogTable state changed. Reason: New leader 48551c20c2a549bab8866b40f9ff3dab. Latest consensus state: current_term: 1 leader_uuid: "48551c20c2a549bab8866b40f9ff3dab" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48551c20c2a549bab8866b40f9ff3dab" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 43725 } } }
I20260924 13:51:29.808857 32134 sys_catalog.cc:458] T 00000000000000000000000000000000 P 48551c20c2a549bab8866b40f9ff3dab [sys.catalog]: This master's current role is: LEADER
I20260924 13:51:29.809037 32141 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 13:51:29.809418 13911 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJN9WRF/build/release/bin/kudu as pid 32061
I20260924 13:51:29.809521 13911 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790257629041645-13911-0/minicluster-data/master-0/wal/instance
I20260924 13:51:29.809577 32141 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 13:51:29.809741 13911 external_mini_cluster.cc:1428] Setting key 268ba82c56183ca7ce0aecf34e7e618f
I20260924 13:51:29.811374 32141 catalog_manager.cc:1347] Generated new cluster ID: f5f19abf47894475ad936024a8c52eb5
I20260924 13:51:29.811427 32141 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 24 13:51:29 dist-test-slave-sjl2 krb5kdc[32048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257889, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.149.254@KRBTEST.COM
I20260924 13:51:29.818320 32141 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 13:51:29.818907 32141 catalog_manager.cc:1504] Loading token signing keys...
I20260924 13:51:29.819880 32151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:29.810246 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59108 (local address 127.13.149.254:43725)
0924 13:51:29.810413 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0924 13:51:29.810421 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0924 13:51:29.810453 (+ 32us) server_negotiation.cc:407] Connection header received
0924 13:51:29.810564 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:51:29.810571 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:51:29.810622 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:51:29.810773 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:51:29.811526 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:51:29.812438 (+ 912us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:51:29.813214 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:51:29.813639 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:51:29.813732 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:51:29.816003 (+ 2271us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:51:29.816007 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:51:29.816007 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 13:51:29.816018 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:51:29.818185 (+ 2167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:51:29.818563 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:51:29.818566 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:51:29.818567 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:51:29.818687 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:51:29.818899 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:51:29.818900 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:51:29.818900 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:51:29.819211 (+ 311us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:51:29.819375 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0924 13:51:29.819488 (+ 113us) server_negotiation.cc:299] Negotiation successful
0924 13:51:29.819650 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123,"thread_start_us":70,"threads_started":1}
I20260924 13:51:29.829861 32141 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 48551c20c2a549bab8866b40f9ff3dab: Generated new TSK 0
I20260924 13:51:29.830047 32141 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.13.149.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.149.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.149.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.193.keytab.
Entry for principal kudu/127.13.149.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.193.keytab.
I20260924 13:51:29.855249 13911 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJN9WRF/build/release/bin/kudu
/tmp/dist-test-taskJN9WRF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790257629041645-13911-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790257629041645-13911-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790257629041645-13911-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790257629041645-13911-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.149.193:0
--local_ip_for_outbound_sockets=127.13.149.193
--webserver_interface=127.13.149.193
--webserver_port=0
--tserver_master_addrs=127.13.149.254:43725
--keytab_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.193.keytab
--principal=kudu/127.13.149.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.149.212:41733
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790257629041645-13911-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790257629041645-13911-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790257629041645-13911-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 13:51:29.935468 32155 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 13:51:29.935688 32155 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 13:51:29.935714 32155 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 13:51:29.936866 32155 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 13:51:29.936910 32155 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 13:51:29.936959 32155 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.149.193
I20260924 13:51:29.938411 32155 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.149.212:41733
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790257629041645-13911-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790257629041645-13911-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.193.keytab
--principal=kudu/127.13.149.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.149.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790257629041645-13911-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.149.193
--webserver_port=0
--tserver_master_addrs=127.13.149.254:43725
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32155
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.149.193
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790257629041645-13911-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 12:29:37 UTC on ccycloud.quasar-iqzbxn.root.comops.site
build id 4670106
I20260924 13:51:29.938607 32155 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 13:51:29.938800 32155 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 13:51:29.943753 32161 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 13:51:29.943776 32160 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 13:51:29.943781 32163 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 13:51:29.943799 32155 server_base.cc:1034] running on GCE node
I20260924 13:51:29.944125 32155 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 13:51:29.944337 32155 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 13:51:29.945472 32155 hybrid_clock.cc:648] HybridClock initialized: now 1790257889945456 us; error 28 us; skew 500 ppm
Sep 24 13:51:29 dist-test-slave-sjl2 krb5kdc[32048](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257889, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.149.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 13:51:29.947897 32155 init.cc:377] Logged in from keytab as kudu/127.13.149.193@KRBTEST.COM (short username kudu)
I20260924 13:51:29.948319 32155 webserver.cc:460] Webserver started at http://127.13.149.193:39607/ using document root <none> and password file <none>
I20260924 13:51:29.948494 32155 fs_manager.cc:359] Metadata directory not provided
I20260924 13:51:29.948532 32155 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 13:51:29.948621 32155 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 13:51:29.949712 32155 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790257629041645-13911-0/minicluster-data/ts-0/data/instance:
uuid: "5519c1503daa4d9e91c1b9960b1eecf1"
format_stamp: "Formatted at 2026-09-24 13:51:29 on dist-test-slave-sjl2"
server_key: "cb61716232515ece54ed880719f6615d"
server_key_iv: "bfc609de74f024acd722cf88cf29cca5"
server_key_version: "encryptionkey@0"
I20260924 13:51:29.950011 32155 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790257629041645-13911-0/minicluster-data/ts-0/wal/instance:
uuid: "5519c1503daa4d9e91c1b9960b1eecf1"
format_stamp: "Formatted at 2026-09-24 13:51:29 on dist-test-slave-sjl2"
server_key: "cb61716232515ece54ed880719f6615d"
server_key_iv: "bfc609de74f024acd722cf88cf29cca5"
server_key_version: "encryptionkey@0"
I20260924 13:51:29.951228 32155 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260924 13:51:29.951948 32170 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 13:51:29.952108 32155 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 13:51:29.952178 32155 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790257629041645-13911-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790257629041645-13911-0/minicluster-data/ts-0/wal
uuid: "5519c1503daa4d9e91c1b9960b1eecf1"
format_stamp: "Formatted at 2026-09-24 13:51:29 on dist-test-slave-sjl2"
server_key: "cb61716232515ece54ed880719f6615d"
server_key_iv: "bfc609de74f024acd722cf88cf29cca5"
server_key_version: "encryptionkey@0"
I20260924 13:51:29.952229 32155 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790257629041645-13911-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790257629041645-13911-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790257629041645-13911-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 13:51:29.966256 32155 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 13:51:29.968320 32155 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 13:51:29.968427 32155 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 13:51:29.968616 32155 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 13:51:29.968906 32155 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 13:51:29.968955 32155 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 13:51:29.968986 32155 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 13:51:29.969007 32155 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 13:51:29.974543 32155 rpc_server.cc:307] RPC server started. Bound to: 127.13.149.193:41815
I20260924 13:51:29.974583 32283 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.149.193:41815 every 8 connection(s)
I20260924 13:51:29.974888 32155 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790257629041645-13911-0/minicluster-data/ts-0/data/info.pb
I20260924 13:51:29.979944 13911 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJN9WRF/build/release/bin/kudu as pid 32155
I20260924 13:51:29.980065 13911 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790257629041645-13911-0/minicluster-data/ts-0/wal/instance
I20260924 13:51:29.980288 13911 external_mini_cluster.cc:1428] Setting key e14b5b48187b74e47ec7a22d33dc4b77
Sep 24 13:51:29 dist-test-slave-sjl2 krb5kdc[32048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257889, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.149.193@KRBTEST.COM for kudu/127.13.149.254@KRBTEST.COM
I20260924 13:51:29.983937 32151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:29.975645 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.149.193:50921 (local address 127.13.149.254:43725)
0924 13:51:29.975693 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0924 13:51:29.975695 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:51:29.975860 (+ 165us) server_negotiation.cc:407] Connection header received
0924 13:51:29.976339 (+ 479us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:51:29.976340 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:51:29.976350 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:51:29.976411 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:51:29.977346 (+ 935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:51:29.977869 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:51:29.978881 (+ 1012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:51:29.979083 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:51:29.979136 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:51:29.981454 (+ 2318us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:51:29.981455 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:51:29.981456 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 13:51:29.981463 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:51:29.982663 (+ 1200us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:51:29.983077 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:51:29.983078 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:51:29.983079 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:51:29.983132 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:51:29.983370 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:51:29.983370 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:51:29.983371 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:51:29.983469 (+ 98us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:51:29.983514 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0924 13:51:29.983802 (+ 288us) server_negotiation.cc:299] Negotiation successful
0924 13:51:29.983871 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260924 13:51:29.984185 32286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:29.975580 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:43725 (local address 127.13.149.193:50921)
0924 13:51:29.975741 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0924 13:51:29.975782 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0924 13:51:29.976264 (+ 482us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:51:29.976461 (+ 197us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:51:29.976464 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:51:29.977059 (+ 595us) client_negotiation.cc:189] Negotiated authn=SASL
0924 13:51:29.977259 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:51:29.977264 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:51:29.977977 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:51:29.977978 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:51:29.978801 (+ 823us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:51:29.978804 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:51:29.979148 (+ 344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:51:29.979149 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:51:29.979243 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:51:29.979762 (+ 519us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 13:51:29.979775 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0924 13:51:29.981361 (+ 1586us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 13:51:29.982739 (+ 1378us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 13:51:29.982745 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 13:51:29.982748 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0924 13:51:29.983016 (+ 268us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 13:51:29.983190 (+ 174us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 13:51:29.983191 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 13:51:29.983191 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 13:51:29.983313 (+ 122us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 13:51:29.983516 (+ 203us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 13:51:29.983520 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 13:51:29.983680 (+ 160us) client_negotiation.cc:769] Sending connection context
0924 13:51:29.983786 (+ 106us) client_negotiation.cc:240] Negotiation successful
0924 13:51:29.983949 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":81,"threads_started":1}
I20260924 13:51:29.984760 32284 heartbeater.cc:347] Connected to a master server at 127.13.149.254:43725
I20260924 13:51:29.984864 32284 heartbeater.cc:464] Registering TS with master...
I20260924 13:51:29.985064 32284 heartbeater.cc:511] Master 127.13.149.254:43725 requested a full tablet report, sending...
I20260924 13:51:29.985594 32093 ts_manager.cc:194] Registered new tserver with Master: 5519c1503daa4d9e91c1b9960b1eecf1 (127.13.149.193:41815)
I20260924 13:51:29.986212 32093 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.149.193@KRBTEST.COM'} at 127.13.149.193:50921
WARNING: no policy specified for kudu/127.13.149.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.149.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.149.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.194.keytab.
Entry for principal kudu/127.13.149.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.194.keytab.
I20260924 13:51:30.007818 13911 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJN9WRF/build/release/bin/kudu
/tmp/dist-test-taskJN9WRF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790257629041645-13911-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790257629041645-13911-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790257629041645-13911-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790257629041645-13911-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.149.194:0
--local_ip_for_outbound_sockets=127.13.149.194
--webserver_interface=127.13.149.194
--webserver_port=0
--tserver_master_addrs=127.13.149.254:43725
--keytab_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.194.keytab
--principal=kudu/127.13.149.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.149.212:41733
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790257629041645-13911-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790257629041645-13911-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790257629041645-13911-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 13:51:30.088768 32289 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 13:51:30.089028 32289 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 13:51:30.089066 32289 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 13:51:30.090255 32289 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 13:51:30.090299 32289 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 13:51:30.090337 32289 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.149.194
I20260924 13:51:30.091784 32289 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.149.212:41733
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790257629041645-13911-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790257629041645-13911-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.194.keytab
--principal=kudu/127.13.149.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.149.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790257629041645-13911-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.149.194
--webserver_port=0
--tserver_master_addrs=127.13.149.254:43725
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32289
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.149.194
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790257629041645-13911-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 12:29:37 UTC on ccycloud.quasar-iqzbxn.root.comops.site
build id 4670106
I20260924 13:51:30.092027 32289 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 13:51:30.092252 32289 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 13:51:30.097210 32294 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 13:51:30.097206 32297 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 13:51:30.097307 32295 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 13:51:30.097393 32289 server_base.cc:1034] running on GCE node
I20260924 13:51:30.097579 32289 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 13:51:30.097813 32289 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 13:51:30.098956 32289 hybrid_clock.cc:648] HybridClock initialized: now 1790257890098944 us; error 32 us; skew 500 ppm
Sep 24 13:51:30 dist-test-slave-sjl2 krb5kdc[32048](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257890, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.149.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 13:51:30.101475 32289 init.cc:377] Logged in from keytab as kudu/127.13.149.194@KRBTEST.COM (short username kudu)
I20260924 13:51:30.101914 32289 webserver.cc:460] Webserver started at http://127.13.149.194:38979/ using document root <none> and password file <none>
I20260924 13:51:30.102114 32289 fs_manager.cc:359] Metadata directory not provided
I20260924 13:51:30.102170 32289 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 13:51:30.102295 32289 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 13:51:30.103348 32289 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790257629041645-13911-0/minicluster-data/ts-1/data/instance:
uuid: "74e71afd01564a22af6a83738c996a29"
format_stamp: "Formatted at 2026-09-24 13:51:30 on dist-test-slave-sjl2"
server_key: "00600ff16c57246d11c14a9303b440a1"
server_key_iv: "188e1de90efebe0a0a9b62554a56e4d2"
server_key_version: "encryptionkey@0"
I20260924 13:51:30.103682 32289 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790257629041645-13911-0/minicluster-data/ts-1/wal/instance:
uuid: "74e71afd01564a22af6a83738c996a29"
format_stamp: "Formatted at 2026-09-24 13:51:30 on dist-test-slave-sjl2"
server_key: "00600ff16c57246d11c14a9303b440a1"
server_key_iv: "188e1de90efebe0a0a9b62554a56e4d2"
server_key_version: "encryptionkey@0"
I20260924 13:51:30.104882 32289 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260924 13:51:30.105611 32304 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 13:51:30.105763 32289 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 13:51:30.105851 32289 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790257629041645-13911-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790257629041645-13911-0/minicluster-data/ts-1/wal
uuid: "74e71afd01564a22af6a83738c996a29"
format_stamp: "Formatted at 2026-09-24 13:51:30 on dist-test-slave-sjl2"
server_key: "00600ff16c57246d11c14a9303b440a1"
server_key_iv: "188e1de90efebe0a0a9b62554a56e4d2"
server_key_version: "encryptionkey@0"
I20260924 13:51:30.105912 32289 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790257629041645-13911-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790257629041645-13911-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790257629041645-13911-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 13:51:30.124502 32289 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 13:51:30.126649 32289 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 13:51:30.126791 32289 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 13:51:30.127003 32289 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 13:51:30.127298 32289 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 13:51:30.127333 32289 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 13:51:30.127363 32289 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 13:51:30.127388 32289 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 13:51:30.133030 32289 rpc_server.cc:307] RPC server started. Bound to: 127.13.149.194:42405
I20260924 13:51:30.133077 32417 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.149.194:42405 every 8 connection(s)
I20260924 13:51:30.133386 32289 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790257629041645-13911-0/minicluster-data/ts-1/data/info.pb
Sep 24 13:51:30 dist-test-slave-sjl2 krb5kdc[32048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257890, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.149.194@KRBTEST.COM for kudu/127.13.149.254@KRBTEST.COM
I20260924 13:51:30.142652 32151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:30.134071 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.149.194:48901 (local address 127.13.149.254:43725)
0924 13:51:30.134113 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0924 13:51:30.134115 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:51:30.134315 (+ 200us) server_negotiation.cc:407] Connection header received
0924 13:51:30.134830 (+ 515us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:51:30.134832 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:51:30.134843 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:51:30.134908 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:51:30.135860 (+ 952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:51:30.136453 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:51:30.137505 (+ 1052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:51:30.137715 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:51:30.137779 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:51:30.140033 (+ 2254us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:51:30.140035 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:51:30.140035 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 13:51:30.140043 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:51:30.141453 (+ 1410us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:51:30.141844 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:51:30.141845 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:51:30.141845 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:51:30.141907 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:51:30.142114 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:51:30.142116 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:51:30.142116 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:51:30.142231 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:51:30.142285 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0924 13:51:30.142496 (+ 211us) server_negotiation.cc:299] Negotiation successful
0924 13:51:30.142559 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260924 13:51:30.142830 32420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:30.134042 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:43725 (local address 127.13.149.194:48901)
0924 13:51:30.134202 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0924 13:51:30.134246 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0924 13:51:30.134719 (+ 473us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:51:30.134978 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:51:30.134981 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:51:30.135557 (+ 576us) client_negotiation.cc:189] Negotiated authn=SASL
0924 13:51:30.135756 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:51:30.135761 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:51:30.136564 (+ 803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:51:30.136564 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:51:30.137410 (+ 846us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:51:30.137413 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:51:30.137806 (+ 393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:51:30.137806 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:51:30.137925 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:51:30.138423 (+ 498us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 13:51:30.138432 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0924 13:51:30.139913 (+ 1481us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 13:51:30.141535 (+ 1622us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 13:51:30.141542 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 13:51:30.141544 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 13:51:30.141762 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 13:51:30.141966 (+ 204us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 13:51:30.141967 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 13:51:30.141967 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 13:51:30.142058 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 13:51:30.142279 (+ 221us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 13:51:30.142282 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 13:51:30.142401 (+ 119us) client_negotiation.cc:769] Sending connection context
0924 13:51:30.142478 (+ 77us) client_negotiation.cc:240] Negotiation successful
0924 13:51:30.142600 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":69,"threads_started":1}
I20260924 13:51:30.142979 13911 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJN9WRF/build/release/bin/kudu as pid 32289
I20260924 13:51:30.143093 13911 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790257629041645-13911-0/minicluster-data/ts-1/wal/instance
I20260924 13:51:30.143275 32418 heartbeater.cc:347] Connected to a master server at 127.13.149.254:43725
I20260924 13:51:30.143340 13911 external_mini_cluster.cc:1428] Setting key 2a4a25db467d0e473beb60b9299e6a8b
I20260924 13:51:30.143388 32418 heartbeater.cc:464] Registering TS with master...
I20260924 13:51:30.143571 32418 heartbeater.cc:511] Master 127.13.149.254:43725 requested a full tablet report, sending...
I20260924 13:51:30.143963 32093 ts_manager.cc:194] Registered new tserver with Master: 74e71afd01564a22af6a83738c996a29 (127.13.149.194:42405)
I20260924 13:51:30.144363 32093 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.149.194@KRBTEST.COM'} at 127.13.149.194:48901
WARNING: no policy specified for kudu/127.13.149.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.149.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.149.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.195.keytab.
Entry for principal kudu/127.13.149.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.195.keytab.
I20260924 13:51:30.166054 13911 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJN9WRF/build/release/bin/kudu
/tmp/dist-test-taskJN9WRF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790257629041645-13911-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790257629041645-13911-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790257629041645-13911-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790257629041645-13911-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.149.195:0
--local_ip_for_outbound_sockets=127.13.149.195
--webserver_interface=127.13.149.195
--webserver_port=0
--tserver_master_addrs=127.13.149.254:43725
--keytab_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.195.keytab
--principal=kudu/127.13.149.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.149.212:41733
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790257629041645-13911-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790257629041645-13911-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790257629041645-13911-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 13:51:30.245697 32423 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 13:51:30.245898 32423 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 13:51:30.245919 32423 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 13:51:30.246995 32423 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 13:51:30.247028 32423 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 13:51:30.247052 32423 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.149.195
I20260924 13:51:30.248435 32423 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.149.212:41733
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790257629041645-13911-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790257629041645-13911-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.195.keytab
--principal=kudu/127.13.149.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.149.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790257629041645-13911-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.149.195
--webserver_port=0
--tserver_master_addrs=127.13.149.254:43725
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32423
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.149.195
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790257629041645-13911-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 12:29:37 UTC on ccycloud.quasar-iqzbxn.root.comops.site
build id 4670106
I20260924 13:51:30.248628 32423 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 13:51:30.248800 32423 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 13:51:30.253605 32431 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 13:51:30.253664 32423 server_base.cc:1034] running on GCE node
W20260924 13:51:30.253580 32429 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 13:51:30.253585 32428 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 13:51:30.254024 32423 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 13:51:30.254274 32423 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 13:51:30.255421 32423 hybrid_clock.cc:648] HybridClock initialized: now 1790257890255406 us; error 29 us; skew 500 ppm
Sep 24 13:51:30 dist-test-slave-sjl2 krb5kdc[32048](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257890, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.149.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 13:51:30.257885 32423 init.cc:377] Logged in from keytab as kudu/127.13.149.195@KRBTEST.COM (short username kudu)
I20260924 13:51:30.258278 32423 webserver.cc:460] Webserver started at http://127.13.149.195:38971/ using document root <none> and password file <none>
I20260924 13:51:30.258466 32423 fs_manager.cc:359] Metadata directory not provided
I20260924 13:51:30.258503 32423 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 13:51:30.258592 32423 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 13:51:30.259585 32423 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790257629041645-13911-0/minicluster-data/ts-2/data/instance:
uuid: "9e293413df4a44dfa8dc10b9c29a3564"
format_stamp: "Formatted at 2026-09-24 13:51:30 on dist-test-slave-sjl2"
server_key: "6e7abafdea3e2a533f86584f5d8806f1"
server_key_iv: "a10208f923790d9e472b763df7bc8628"
server_key_version: "encryptionkey@0"
I20260924 13:51:30.259869 32423 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790257629041645-13911-0/minicluster-data/ts-2/wal/instance:
uuid: "9e293413df4a44dfa8dc10b9c29a3564"
format_stamp: "Formatted at 2026-09-24 13:51:30 on dist-test-slave-sjl2"
server_key: "6e7abafdea3e2a533f86584f5d8806f1"
server_key_iv: "a10208f923790d9e472b763df7bc8628"
server_key_version: "encryptionkey@0"
I20260924 13:51:30.261101 32423 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260924 13:51:30.261709 32438 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 13:51:30.261854 32423 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260924 13:51:30.261929 32423 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790257629041645-13911-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790257629041645-13911-0/minicluster-data/ts-2/wal
uuid: "9e293413df4a44dfa8dc10b9c29a3564"
format_stamp: "Formatted at 2026-09-24 13:51:30 on dist-test-slave-sjl2"
server_key: "6e7abafdea3e2a533f86584f5d8806f1"
server_key_iv: "a10208f923790d9e472b763df7bc8628"
server_key_version: "encryptionkey@0"
I20260924 13:51:30.262002 32423 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790257629041645-13911-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790257629041645-13911-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790257629041645-13911-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 13:51:30.272666 32423 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 13:51:30.274641 32423 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 13:51:30.274814 32423 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 13:51:30.275032 32423 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 13:51:30.275328 32423 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 13:51:30.275359 32423 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 13:51:30.275390 32423 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 13:51:30.275413 32423 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 13:51:30.280853 32423 rpc_server.cc:307] RPC server started. Bound to: 127.13.149.195:42337
I20260924 13:51:30.280885 32551 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.149.195:42337 every 8 connection(s)
I20260924 13:51:30.281229 32423 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790257629041645-13911-0/minicluster-data/ts-2/data/info.pb
Sep 24 13:51:30 dist-test-slave-sjl2 krb5kdc[32048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257890, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.149.195@KRBTEST.COM for kudu/127.13.149.254@KRBTEST.COM
I20260924 13:51:30.289975 32151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:30.281929 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.149.195:50047 (local address 127.13.149.254:43725)
0924 13:51:30.281985 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0924 13:51:30.281987 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:51:30.282161 (+ 174us) server_negotiation.cc:407] Connection header received
0924 13:51:30.282694 (+ 533us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:51:30.282695 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:51:30.282707 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:51:30.282757 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:51:30.283701 (+ 944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:51:30.284201 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:51:30.285227 (+ 1026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:51:30.285453 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:51:30.285505 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:51:30.287508 (+ 2003us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:51:30.287509 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:51:30.287510 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 13:51:30.287517 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:51:30.288716 (+ 1199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:51:30.289181 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:51:30.289182 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:51:30.289182 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:51:30.289229 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:51:30.289430 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:51:30.289430 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:51:30.289430 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:51:30.289526 (+ 96us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:51:30.289569 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0924 13:51:30.289853 (+ 284us) server_negotiation.cc:299] Negotiation successful
0924 13:51:30.289911 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260924 13:51:30.290233 32554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:30.281886 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:43725 (local address 127.13.149.195:50047)
0924 13:51:30.282042 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0924 13:51:30.282081 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0924 13:51:30.282569 (+ 488us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:51:30.282838 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:51:30.282842 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:51:30.283402 (+ 560us) client_negotiation.cc:189] Negotiated authn=SASL
0924 13:51:30.283596 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:51:30.283610 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:51:30.284282 (+ 672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:51:30.284283 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:51:30.285148 (+ 865us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:51:30.285151 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:51:30.285533 (+ 382us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:51:30.285534 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:51:30.285672 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:51:30.286120 (+ 448us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 13:51:30.286128 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0924 13:51:30.287425 (+ 1297us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 13:51:30.288791 (+ 1366us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 13:51:30.288797 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 13:51:30.288799 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 13:51:30.289075 (+ 276us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 13:51:30.289297 (+ 222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 13:51:30.289297 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 13:51:30.289298 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 13:51:30.289395 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 13:51:30.289610 (+ 215us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 13:51:30.289617 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 13:51:30.289741 (+ 124us) client_negotiation.cc:769] Sending connection context
0924 13:51:30.289842 (+ 101us) client_negotiation.cc:240] Negotiation successful
0924 13:51:30.289968 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":74,"threads_started":1}
I20260924 13:51:30.290678 32552 heartbeater.cc:347] Connected to a master server at 127.13.149.254:43725
I20260924 13:51:30.290769 32552 heartbeater.cc:464] Registering TS with master...
I20260924 13:51:30.290815 13911 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJN9WRF/build/release/bin/kudu as pid 32423
I20260924 13:51:30.290931 13911 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790257629041645-13911-0/minicluster-data/ts-2/wal/instance
I20260924 13:51:30.290953 32552 heartbeater.cc:511] Master 127.13.149.254:43725 requested a full tablet report, sending...
I20260924 13:51:30.291107 13911 external_mini_cluster.cc:1428] Setting key 445090d7c014007915ac726577a22cdb
I20260924 13:51:30.291318 32093 ts_manager.cc:194] Registered new tserver with Master: 9e293413df4a44dfa8dc10b9c29a3564 (127.13.149.195:42337)
I20260924 13:51:30.291749 32093 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.149.195@KRBTEST.COM'} at 127.13.149.195:50047
I20260924 13:51:30.291967 13911 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260924 13:51:30.299228 32151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:30.293401 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59112 (local address 127.13.149.254:43725)
0924 13:51:30.293462 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0924 13:51:30.293464 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:51:30.293504 (+ 40us) server_negotiation.cc:407] Connection header received
0924 13:51:30.293556 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:51:30.293557 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:51:30.293564 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:51:30.293605 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:51:30.294118 (+ 513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:51:30.294586 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:51:30.295297 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:51:30.295471 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:51:30.295513 (+ 42us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:51:30.296795 (+ 1282us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:51:30.296797 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:51:30.296797 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 13:51:30.296804 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:51:30.298123 (+ 1319us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:51:30.298523 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:51:30.298524 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:51:30.298524 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:51:30.298574 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:51:30.298742 (+ 168us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:51:30.298742 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:51:30.298743 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:51:30.298872 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:51:30.298947 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0924 13:51:30.299105 (+ 158us) server_negotiation.cc:299] Negotiation successful
0924 13:51:30.299152 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260924 13:51:30.301023 32093 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59112:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 13:51:30.302594 32093 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 13:51:30.308907 32562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:30.304826 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51982 (local address 127.13.149.194:42405)
0924 13:51:30.305193 (+ 367us) server_negotiation.cc:206] Beginning negotiation
0924 13:51:30.305197 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 13:51:30.305272 (+ 75us) server_negotiation.cc:407] Connection header received
0924 13:51:30.305346 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:51:30.305348 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:51:30.305365 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:51:30.305439 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 13:51:30.305690 (+ 251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:51:30.306504 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:51:30.307820 (+ 1316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:51:30.308446 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:51:30.308509 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:51:30.308568 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0924 13:51:30.308769 (+ 201us) server_negotiation.cc:299] Negotiation successful
0924 13:51:30.308820 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":331,"thread_start_us":81,"threads_started":1}
I20260924 13:51:30.309103 32565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:30.305002 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49014 (local address 127.13.149.193:41815)
0924 13:51:30.305264 (+ 262us) server_negotiation.cc:206] Beginning negotiation
0924 13:51:30.305268 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 13:51:30.305405 (+ 137us) server_negotiation.cc:407] Connection header received
0924 13:51:30.305458 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:51:30.305460 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:51:30.305475 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:51:30.305540 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 13:51:30.305727 (+ 187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:51:30.306862 (+ 1135us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:51:30.307856 (+ 994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:51:30.308564 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:51:30.308643 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:51:30.308693 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0924 13:51:30.308978 (+ 285us) server_negotiation.cc:299] Negotiation successful
0924 13:51:30.309022 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":234,"thread_start_us":74,"threads_started":1}
I20260924 13:51:30.308894 32560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:30.304649 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.194:42405 (local address 127.0.0.1:51982)
0924 13:51:30.305119 (+ 470us) negotiation.cc:107] Waiting for socket to connect
0924 13:51:30.305173 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0924 13:51:30.305253 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:51:30.305521 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:51:30.305522 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:51:30.305524 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 13:51:30.305576 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:51:30.305577 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:51:30.306599 (+ 1022us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:51:30.306600 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:51:30.307726 (+ 1126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:51:30.307729 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:51:30.308549 (+ 820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:51:30.308550 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:51:30.308630 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:51:30.308674 (+ 44us) client_negotiation.cc:769] Sending connection context
0924 13:51:30.308750 (+ 76us) client_negotiation.cc:240] Negotiation successful
0924 13:51:30.308805 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":442,"thread_start_us":65,"threads_started":1}
I20260924 13:51:30.309399 32564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:30.304993 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43606 (local address 127.13.149.195:42337)
0924 13:51:30.305172 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0924 13:51:30.305177 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 13:51:30.305353 (+ 176us) server_negotiation.cc:407] Connection header received
0924 13:51:30.305395 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:51:30.305397 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:51:30.305411 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:51:30.305480 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 13:51:30.305676 (+ 196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:51:30.306861 (+ 1185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:51:30.308197 (+ 1336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:51:30.308898 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:51:30.308985 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:51:30.309036 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0924 13:51:30.309276 (+ 240us) server_negotiation.cc:299] Negotiation successful
0924 13:51:30.309320 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":146,"thread_start_us":68,"threads_started":1}
I20260924 13:51:30.309667 32563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:30.304651 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.195:42337 (local address 127.0.0.1:43606)
0924 13:51:30.305216 (+ 565us) negotiation.cc:107] Waiting for socket to connect
0924 13:51:30.305226 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 13:51:30.305283 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:51:30.305483 (+ 200us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:51:30.305486 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:51:30.305488 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 13:51:30.305572 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:51:30.305574 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:51:30.306964 (+ 1390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:51:30.306965 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:51:30.308137 (+ 1172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:51:30.308140 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:51:30.309071 (+ 931us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:51:30.309072 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:51:30.309149 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:51:30.309153 (+ 4us) client_negotiation.cc:769] Sending connection context
0924 13:51:30.309255 (+ 102us) client_negotiation.cc:240] Negotiation successful
0924 13:51:30.309296 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":552,"mutex_wait_us":36,"spinlock_wait_cycles":896,"thread_start_us":69,"threads_started":1}
I20260924 13:51:30.309672 32561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:30.304852 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.193:41815 (local address 127.0.0.1:49014)
0924 13:51:30.305354 (+ 502us) negotiation.cc:107] Waiting for socket to connect
0924 13:51:30.305360 (+ 6us) client_negotiation.cc:174] Beginning negotiation
0924 13:51:30.305396 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:51:30.305610 (+ 214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:51:30.305610 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:51:30.305612 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 13:51:30.305656 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:51:30.305657 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:51:30.306974 (+ 1317us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:51:30.306975 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:51:30.307775 (+ 800us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:51:30.307777 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:51:30.308628 (+ 851us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:51:30.308629 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:51:30.308784 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:51:30.308788 (+ 4us) client_negotiation.cc:769] Sending connection context
0924 13:51:30.308829 (+ 41us) client_negotiation.cc:240] Negotiation successful
0924 13:51:30.308861 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":492,"mutex_wait_us":22,"thread_start_us":55,"threads_started":1}
I20260924 13:51:30.310113 32352 tablet_service.cc:1467] Processing CreateTablet for tablet 48aabacfb90e45f7893abc7011e05307 (DEFAULT_TABLE table=test-table [id=30dbbaba685547b9bb495f8ca80b20aa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 13:51:30.310490 32352 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 48aabacfb90e45f7893abc7011e05307. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 13:51:30.310487 32486 tablet_service.cc:1467] Processing CreateTablet for tablet 48aabacfb90e45f7893abc7011e05307 (DEFAULT_TABLE table=test-table [id=30dbbaba685547b9bb495f8ca80b20aa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 13:51:30.310779 32486 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 48aabacfb90e45f7893abc7011e05307. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 13:51:30.313000 32566 tablet_bootstrap.cc:492] T 48aabacfb90e45f7893abc7011e05307 P 74e71afd01564a22af6a83738c996a29: Bootstrap starting.
I20260924 13:51:30.313577 32566 tablet_bootstrap.cc:654] T 48aabacfb90e45f7893abc7011e05307 P 74e71afd01564a22af6a83738c996a29: Neither blocks nor log segments found. Creating new log.
I20260924 13:51:30.313660 32567 tablet_bootstrap.cc:492] T 48aabacfb90e45f7893abc7011e05307 P 9e293413df4a44dfa8dc10b9c29a3564: Bootstrap starting.
I20260924 13:51:30.313642 32218 tablet_service.cc:1467] Processing CreateTablet for tablet 48aabacfb90e45f7893abc7011e05307 (DEFAULT_TABLE table=test-table [id=30dbbaba685547b9bb495f8ca80b20aa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 13:51:30.313992 32218 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 48aabacfb90e45f7893abc7011e05307. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 13:51:30.314100 32566 log.cc:824] T 48aabacfb90e45f7893abc7011e05307 P 74e71afd01564a22af6a83738c996a29: Log is configured to *not* fsync() on all Append() calls
I20260924 13:51:30.314251 32567 tablet_bootstrap.cc:654] T 48aabacfb90e45f7893abc7011e05307 P 9e293413df4a44dfa8dc10b9c29a3564: Neither blocks nor log segments found. Creating new log.
I20260924 13:51:30.314757 32566 tablet_bootstrap.cc:492] T 48aabacfb90e45f7893abc7011e05307 P 74e71afd01564a22af6a83738c996a29: No bootstrap required, opened a new log
I20260924 13:51:30.314786 32567 log.cc:824] T 48aabacfb90e45f7893abc7011e05307 P 9e293413df4a44dfa8dc10b9c29a3564: Log is configured to *not* fsync() on all Append() calls
I20260924 13:51:30.314816 32566 ts_tablet_manager.cc:1397] T 48aabacfb90e45f7893abc7011e05307 P 74e71afd01564a22af6a83738c996a29: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260924 13:51:30.315455 32567 tablet_bootstrap.cc:492] T 48aabacfb90e45f7893abc7011e05307 P 9e293413df4a44dfa8dc10b9c29a3564: No bootstrap required, opened a new log
I20260924 13:51:30.315523 32567 ts_tablet_manager.cc:1397] T 48aabacfb90e45f7893abc7011e05307 P 9e293413df4a44dfa8dc10b9c29a3564: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260924 13:51:30.316332 32566 raft_consensus.cc:348] T 48aabacfb90e45f7893abc7011e05307 P 74e71afd01564a22af6a83738c996a29 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e293413df4a44dfa8dc10b9c29a3564" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 42337 } } peers { permanent_uuid: "74e71afd01564a22af6a83738c996a29" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 42405 } } peers { permanent_uuid: "5519c1503daa4d9e91c1b9960b1eecf1" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 41815 } }
I20260924 13:51:30.316478 32566 raft_consensus.cc:374] T 48aabacfb90e45f7893abc7011e05307 P 74e71afd01564a22af6a83738c996a29 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 13:51:30.316510 32566 raft_consensus.cc:729] T 48aabacfb90e45f7893abc7011e05307 P 74e71afd01564a22af6a83738c996a29 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 74e71afd01564a22af6a83738c996a29, State: Initialized, Role: FOLLOWER
I20260924 13:51:30.316648 32566 consensus_queue.cc:260] T 48aabacfb90e45f7893abc7011e05307 P 74e71afd01564a22af6a83738c996a29 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e293413df4a44dfa8dc10b9c29a3564" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 42337 } } peers { permanent_uuid: "74e71afd01564a22af6a83738c996a29" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 42405 } } peers { permanent_uuid: "5519c1503daa4d9e91c1b9960b1eecf1" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 41815 } }
I20260924 13:51:30.316955 32418 heartbeater.cc:503] Master 127.13.149.254:43725 was elected leader, sending a full tablet report...
I20260924 13:51:30.316954 32566 ts_tablet_manager.cc:1428] T 48aabacfb90e45f7893abc7011e05307 P 74e71afd01564a22af6a83738c996a29: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260924 13:51:30.317409 32571 tablet_bootstrap.cc:492] T 48aabacfb90e45f7893abc7011e05307 P 5519c1503daa4d9e91c1b9960b1eecf1: Bootstrap starting.
I20260924 13:51:30.317454 32567 raft_consensus.cc:348] T 48aabacfb90e45f7893abc7011e05307 P 9e293413df4a44dfa8dc10b9c29a3564 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e293413df4a44dfa8dc10b9c29a3564" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 42337 } } peers { permanent_uuid: "74e71afd01564a22af6a83738c996a29" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 42405 } } peers { permanent_uuid: "5519c1503daa4d9e91c1b9960b1eecf1" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 41815 } }
I20260924 13:51:30.317610 32567 raft_consensus.cc:374] T 48aabacfb90e45f7893abc7011e05307 P 9e293413df4a44dfa8dc10b9c29a3564 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 13:51:30.317644 32567 raft_consensus.cc:729] T 48aabacfb90e45f7893abc7011e05307 P 9e293413df4a44dfa8dc10b9c29a3564 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e293413df4a44dfa8dc10b9c29a3564, State: Initialized, Role: FOLLOWER
I20260924 13:51:30.317749 32567 consensus_queue.cc:260] T 48aabacfb90e45f7893abc7011e05307 P 9e293413df4a44dfa8dc10b9c29a3564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e293413df4a44dfa8dc10b9c29a3564" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 42337 } } peers { permanent_uuid: "74e71afd01564a22af6a83738c996a29" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 42405 } } peers { permanent_uuid: "5519c1503daa4d9e91c1b9960b1eecf1" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 41815 } }
I20260924 13:51:30.317991 32567 ts_tablet_manager.cc:1428] T 48aabacfb90e45f7893abc7011e05307 P 9e293413df4a44dfa8dc10b9c29a3564: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260924 13:51:30.318019 32552 heartbeater.cc:503] Master 127.13.149.254:43725 was elected leader, sending a full tablet report...
I20260924 13:51:30.318284 32571 tablet_bootstrap.cc:654] T 48aabacfb90e45f7893abc7011e05307 P 5519c1503daa4d9e91c1b9960b1eecf1: Neither blocks nor log segments found. Creating new log.
I20260924 13:51:30.318837 32571 log.cc:824] T 48aabacfb90e45f7893abc7011e05307 P 5519c1503daa4d9e91c1b9960b1eecf1: Log is configured to *not* fsync() on all Append() calls
I20260924 13:51:30.319428 32571 tablet_bootstrap.cc:492] T 48aabacfb90e45f7893abc7011e05307 P 5519c1503daa4d9e91c1b9960b1eecf1: No bootstrap required, opened a new log
I20260924 13:51:30.319478 32571 ts_tablet_manager.cc:1397] T 48aabacfb90e45f7893abc7011e05307 P 5519c1503daa4d9e91c1b9960b1eecf1: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260924 13:51:30.320791 32571 raft_consensus.cc:348] T 48aabacfb90e45f7893abc7011e05307 P 5519c1503daa4d9e91c1b9960b1eecf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e293413df4a44dfa8dc10b9c29a3564" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 42337 } } peers { permanent_uuid: "74e71afd01564a22af6a83738c996a29" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 42405 } } peers { permanent_uuid: "5519c1503daa4d9e91c1b9960b1eecf1" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 41815 } }
I20260924 13:51:30.321004 32571 raft_consensus.cc:374] T 48aabacfb90e45f7893abc7011e05307 P 5519c1503daa4d9e91c1b9960b1eecf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 13:51:30.321058 32571 raft_consensus.cc:729] T 48aabacfb90e45f7893abc7011e05307 P 5519c1503daa4d9e91c1b9960b1eecf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5519c1503daa4d9e91c1b9960b1eecf1, State: Initialized, Role: FOLLOWER
I20260924 13:51:30.321170 32571 consensus_queue.cc:260] T 48aabacfb90e45f7893abc7011e05307 P 5519c1503daa4d9e91c1b9960b1eecf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e293413df4a44dfa8dc10b9c29a3564" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 42337 } } peers { permanent_uuid: "74e71afd01564a22af6a83738c996a29" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 42405 } } peers { permanent_uuid: "5519c1503daa4d9e91c1b9960b1eecf1" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 41815 } }
I20260924 13:51:30.321422 32571 ts_tablet_manager.cc:1428] T 48aabacfb90e45f7893abc7011e05307 P 5519c1503daa4d9e91c1b9960b1eecf1: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260924 13:51:30.321535 32284 heartbeater.cc:503] Master 127.13.149.254:43725 was elected leader, sending a full tablet report...
W20260924 13:51:30.383955 32419 tablet.cc:2369] T 48aabacfb90e45f7893abc7011e05307 P 74e71afd01564a22af6a83738c996a29: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260924 13:51:30.401981 32570 raft_consensus.cc:482] T 48aabacfb90e45f7893abc7011e05307 P 74e71afd01564a22af6a83738c996a29 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 13:51:30.402117 32570 raft_consensus.cc:504] T 48aabacfb90e45f7893abc7011e05307 P 74e71afd01564a22af6a83738c996a29 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e293413df4a44dfa8dc10b9c29a3564" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 42337 } } peers { permanent_uuid: "74e71afd01564a22af6a83738c996a29" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 42405 } } peers { permanent_uuid: "5519c1503daa4d9e91c1b9960b1eecf1" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 41815 } }
I20260924 13:51:30.402495 32570 leader_election.cc:288] T 48aabacfb90e45f7893abc7011e05307 P 74e71afd01564a22af6a83738c996a29 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9e293413df4a44dfa8dc10b9c29a3564 (127.13.149.195:42337), 5519c1503daa4d9e91c1b9960b1eecf1 (127.13.149.193:41815)
I20260924 13:51:30.405380 32564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:30.402734 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.149.194:40381 (local address 127.13.149.195:42337)
0924 13:51:30.402803 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0924 13:51:30.402821 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0924 13:51:30.402998 (+ 177us) server_negotiation.cc:407] Connection header received
0924 13:51:30.403037 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:51:30.403039 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:51:30.403048 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:51:30.403105 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 13:51:30.403251 (+ 146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:51:30.403698 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:51:30.404663 (+ 965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:51:30.405083 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:51:30.405148 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:51:30.405157 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0924 13:51:30.405272 (+ 115us) server_negotiation.cc:299] Negotiation successful
0924 13:51:30.405310 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260924 13:51:30.405380 32565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:30.402796 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.149.194:59471 (local address 127.13.149.193:41815)
0924 13:51:30.402852 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0924 13:51:30.402854 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:51:30.402862 (+ 8us) server_negotiation.cc:407] Connection header received
0924 13:51:30.402891 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:51:30.402892 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:51:30.402901 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:51:30.402965 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 13:51:30.403178 (+ 213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:51:30.403685 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:51:30.404664 (+ 979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:51:30.405066 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:51:30.405131 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:51:30.405141 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0924 13:51:30.405282 (+ 141us) server_negotiation.cc:299] Negotiation successful
0924 13:51:30.405316 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260924 13:51:30.405387 32420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:30.402685 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.193:41815 (local address 127.13.149.194:59471)
0924 13:51:30.402731 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0924 13:51:30.402742 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0924 13:51:30.402790 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:51:30.402970 (+ 180us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:51:30.402970 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:51:30.402972 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 13:51:30.403054 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:51:30.403056 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:51:30.403764 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:51:30.403765 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:51:30.404575 (+ 810us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:51:30.404577 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:51:30.405138 (+ 561us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:51:30.405139 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:51:30.405219 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:51:30.405222 (+ 3us) client_negotiation.cc:769] Sending connection context
0924 13:51:30.405270 (+ 48us) client_negotiation.cc:240] Negotiation successful
0924 13:51:30.405315 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":24,"mutex_wait_us":30}
I20260924 13:51:30.405395 32575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:30.402690 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.195:42337 (local address 127.13.149.194:40381)
0924 13:51:30.402930 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0924 13:51:30.402939 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0924 13:51:30.402994 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:51:30.403114 (+ 120us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:51:30.403114 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:51:30.403116 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 13:51:30.403190 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:51:30.403192 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:51:30.403776 (+ 584us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:51:30.403776 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:51:30.404571 (+ 795us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:51:30.404573 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:51:30.405156 (+ 583us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:51:30.405157 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:51:30.405211 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:51:30.405214 (+ 3us) client_negotiation.cc:769] Sending connection context
0924 13:51:30.405257 (+ 43us) client_negotiation.cc:240] Negotiation successful
0924 13:51:30.405298 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":222,"mutex_wait_us":64,"thread_start_us":124,"threads_started":1}
I20260924 13:51:30.405767 32238 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "48aabacfb90e45f7893abc7011e05307" candidate_uuid: "74e71afd01564a22af6a83738c996a29" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5519c1503daa4d9e91c1b9960b1eecf1" is_pre_election: true
I20260924 13:51:30.405926 32238 raft_consensus.cc:2454] T 48aabacfb90e45f7893abc7011e05307 P 5519c1503daa4d9e91c1b9960b1eecf1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74e71afd01564a22af6a83738c996a29 in term 0.
I20260924 13:51:30.405917 32506 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "48aabacfb90e45f7893abc7011e05307" candidate_uuid: "74e71afd01564a22af6a83738c996a29" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e293413df4a44dfa8dc10b9c29a3564" is_pre_election: true
I20260924 13:51:30.406064 32506 raft_consensus.cc:2454] T 48aabacfb90e45f7893abc7011e05307 P 9e293413df4a44dfa8dc10b9c29a3564 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74e71afd01564a22af6a83738c996a29 in term 0.
I20260924 13:51:30.406199 32308 leader_election.cc:302] T 48aabacfb90e45f7893abc7011e05307 P 74e71afd01564a22af6a83738c996a29 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5519c1503daa4d9e91c1b9960b1eecf1, 74e71afd01564a22af6a83738c996a29; no voters:
I20260924 13:51:30.406347 32570 raft_consensus.cc:2781] T 48aabacfb90e45f7893abc7011e05307 P 74e71afd01564a22af6a83738c996a29 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 13:51:30.406405 32570 raft_consensus.cc:482] T 48aabacfb90e45f7893abc7011e05307 P 74e71afd01564a22af6a83738c996a29 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 13:51:30.406428 32570 raft_consensus.cc:3037] T 48aabacfb90e45f7893abc7011e05307 P 74e71afd01564a22af6a83738c996a29 [term 0 FOLLOWER]: Advancing to term 1
I20260924 13:51:30.407078 32570 raft_consensus.cc:504] T 48aabacfb90e45f7893abc7011e05307 P 74e71afd01564a22af6a83738c996a29 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e293413df4a44dfa8dc10b9c29a3564" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 42337 } } peers { permanent_uuid: "74e71afd01564a22af6a83738c996a29" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 42405 } } peers { permanent_uuid: "5519c1503daa4d9e91c1b9960b1eecf1" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 41815 } }
I20260924 13:51:30.407191 32570 leader_election.cc:288] T 48aabacfb90e45f7893abc7011e05307 P 74e71afd01564a22af6a83738c996a29 [CANDIDATE]: Term 1 election: Requested vote from peers 9e293413df4a44dfa8dc10b9c29a3564 (127.13.149.195:42337), 5519c1503daa4d9e91c1b9960b1eecf1 (127.13.149.193:41815)
I20260924 13:51:30.407356 32238 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "48aabacfb90e45f7893abc7011e05307" candidate_uuid: "74e71afd01564a22af6a83738c996a29" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5519c1503daa4d9e91c1b9960b1eecf1"
I20260924 13:51:30.407356 32506 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "48aabacfb90e45f7893abc7011e05307" candidate_uuid: "74e71afd01564a22af6a83738c996a29" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e293413df4a44dfa8dc10b9c29a3564"
I20260924 13:51:30.407433 32506 raft_consensus.cc:3037] T 48aabacfb90e45f7893abc7011e05307 P 9e293413df4a44dfa8dc10b9c29a3564 [term 0 FOLLOWER]: Advancing to term 1
I20260924 13:51:30.407433 32238 raft_consensus.cc:3037] T 48aabacfb90e45f7893abc7011e05307 P 5519c1503daa4d9e91c1b9960b1eecf1 [term 0 FOLLOWER]: Advancing to term 1
I20260924 13:51:30.408317 32238 raft_consensus.cc:2454] T 48aabacfb90e45f7893abc7011e05307 P 5519c1503daa4d9e91c1b9960b1eecf1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74e71afd01564a22af6a83738c996a29 in term 1.
I20260924 13:51:30.408311 32506 raft_consensus.cc:2454] T 48aabacfb90e45f7893abc7011e05307 P 9e293413df4a44dfa8dc10b9c29a3564 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74e71afd01564a22af6a83738c996a29 in term 1.
I20260924 13:51:30.408457 32308 leader_election.cc:302] T 48aabacfb90e45f7893abc7011e05307 P 74e71afd01564a22af6a83738c996a29 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5519c1503daa4d9e91c1b9960b1eecf1, 74e71afd01564a22af6a83738c996a29; no voters:
I20260924 13:51:30.408562 32570 raft_consensus.cc:2781] T 48aabacfb90e45f7893abc7011e05307 P 74e71afd01564a22af6a83738c996a29 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 13:51:30.408725 32570 raft_consensus.cc:686] T 48aabacfb90e45f7893abc7011e05307 P 74e71afd01564a22af6a83738c996a29 [term 1 LEADER]: Becoming Leader. State: Replica: 74e71afd01564a22af6a83738c996a29, State: Running, Role: LEADER
I20260924 13:51:30.408833 32570 consensus_queue.cc:237] T 48aabacfb90e45f7893abc7011e05307 P 74e71afd01564a22af6a83738c996a29 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e293413df4a44dfa8dc10b9c29a3564" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 42337 } } peers { permanent_uuid: "74e71afd01564a22af6a83738c996a29" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 42405 } } peers { permanent_uuid: "5519c1503daa4d9e91c1b9960b1eecf1" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 41815 } }
I20260924 13:51:30.409678 32093 catalog_manager.cc:5495] T 48aabacfb90e45f7893abc7011e05307 P 74e71afd01564a22af6a83738c996a29 reported cstate change: term changed from 0 to 1, leader changed from <none> to 74e71afd01564a22af6a83738c996a29 (127.13.149.194). New cstate: current_term: 1 leader_uuid: "74e71afd01564a22af6a83738c996a29" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e293413df4a44dfa8dc10b9c29a3564" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 42337 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "74e71afd01564a22af6a83738c996a29" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 42405 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5519c1503daa4d9e91c1b9960b1eecf1" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 41815 } health_report { overall_health: UNKNOWN } } }
I20260924 13:51:30.426950 32562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:30.424649 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51986 (local address 127.13.149.194:42405)
0924 13:51:30.424710 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0924 13:51:30.424712 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:51:30.424721 (+ 9us) server_negotiation.cc:407] Connection header received
0924 13:51:30.424803 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:51:30.424804 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:51:30.424822 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:51:30.424882 (+ 60us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 13:51:30.425021 (+ 139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:51:30.425503 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:51:30.426237 (+ 734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:51:30.426425 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:51:30.426479 (+ 54us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:51:30.426624 (+ 145us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 13:51:30.426713 (+ 89us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 13:51:30.426796 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0924 13:51:30.426843 (+ 47us) server_negotiation.cc:299] Negotiation successful
0924 13:51:30.426854 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260924 13:51:30.428619 32238 raft_consensus.cc:1261] T 48aabacfb90e45f7893abc7011e05307 P 5519c1503daa4d9e91c1b9960b1eecf1 [term 1 FOLLOWER]: Refusing update from remote peer 74e71afd01564a22af6a83738c996a29: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 13:51:30.428629 32506 raft_consensus.cc:1261] T 48aabacfb90e45f7893abc7011e05307 P 9e293413df4a44dfa8dc10b9c29a3564 [term 1 FOLLOWER]: Refusing update from remote peer 74e71afd01564a22af6a83738c996a29: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 13:51:30.429003 32576 consensus_queue.cc:1035] T 48aabacfb90e45f7893abc7011e05307 P 74e71afd01564a22af6a83738c996a29 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5519c1503daa4d9e91c1b9960b1eecf1" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 41815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 13:51:30.429134 32570 consensus_queue.cc:1035] T 48aabacfb90e45f7893abc7011e05307 P 74e71afd01564a22af6a83738c996a29 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e293413df4a44dfa8dc10b9c29a3564" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 42337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 13:51:30.431488 32579 mvcc.cc:205] Tried to move back new op lower bound from 7332896319193190400 to 7332896319115268096. Current Snapshot: MvccSnapshot[applied={T|T < 7332896319193190400}]
I20260924 13:51:30.432520 32580 mvcc.cc:205] Tried to move back new op lower bound from 7332896319193190400 to 7332896319115268096. Current Snapshot: MvccSnapshot[applied={T|T < 7332896319193190400}]
I20260924 13:51:30.433845 32092 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59112:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 13:51:30.433940 32092 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59112:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 13:51:30.434731 32092 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 48551c20c2a549bab8866b40f9ff3dab: Sending DeleteTablet for 3 replicas of tablet 48aabacfb90e45f7893abc7011e05307
I20260924 13:51:30.435075 32486 tablet_service.cc:1514] Processing DeleteTablet for tablet 48aabacfb90e45f7893abc7011e05307 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 13:51:30 UTC) from {username='kudu', principal='kudu/127.13.149.254@KRBTEST.COM'} at 127.0.0.1:43606
I20260924 13:51:30.435097 32352 tablet_service.cc:1514] Processing DeleteTablet for tablet 48aabacfb90e45f7893abc7011e05307 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 13:51:30 UTC) from {username='kudu', principal='kudu/127.13.149.254@KRBTEST.COM'} at 127.0.0.1:51982
I20260924 13:51:30.435123 32218 tablet_service.cc:1514] Processing DeleteTablet for tablet 48aabacfb90e45f7893abc7011e05307 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 13:51:30 UTC) from {username='kudu', principal='kudu/127.13.149.254@KRBTEST.COM'} at 127.0.0.1:49014
I20260924 13:51:30.435341 32591 tablet_replica.cc:330] T 48aabacfb90e45f7893abc7011e05307 P 74e71afd01564a22af6a83738c996a29: stopping tablet replica
I20260924 13:51:30.435436 32591 raft_consensus.cc:2229] T 48aabacfb90e45f7893abc7011e05307 P 74e71afd01564a22af6a83738c996a29 [term 1 LEADER]: Raft consensus shutting down.
I20260924 13:51:30.435534 32591 raft_consensus.cc:2258] T 48aabacfb90e45f7893abc7011e05307 P 74e71afd01564a22af6a83738c996a29 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 13:51:30.435619 32589 tablet_replica.cc:330] T 48aabacfb90e45f7893abc7011e05307 P 5519c1503daa4d9e91c1b9960b1eecf1: stopping tablet replica
I20260924 13:51:30.435644 13911 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJN9WRF/build/release/bin/kudu with pid 32155
I20260924 13:51:30.437676 32591 ts_tablet_manager.cc:1905] T 48aabacfb90e45f7893abc7011e05307 P 74e71afd01564a22af6a83738c996a29: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 13:51:30.437731 32590 tablet_replica.cc:330] T 48aabacfb90e45f7893abc7011e05307 P 9e293413df4a44dfa8dc10b9c29a3564: stopping tablet replica
I20260924 13:51:30.437832 32590 raft_consensus.cc:2229] T 48aabacfb90e45f7893abc7011e05307 P 9e293413df4a44dfa8dc10b9c29a3564 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 13:51:30.437942 32590 raft_consensus.cc:2258] T 48aabacfb90e45f7893abc7011e05307 P 9e293413df4a44dfa8dc10b9c29a3564 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 13:51:30.438335 32590 ts_tablet_manager.cc:1905] T 48aabacfb90e45f7893abc7011e05307 P 9e293413df4a44dfa8dc10b9c29a3564: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 13:51:30.439687 32591 ts_tablet_manager.cc:1918] T 48aabacfb90e45f7893abc7011e05307 P 74e71afd01564a22af6a83738c996a29: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 13:51:30.439813 32591 log.cc:1196] T 48aabacfb90e45f7893abc7011e05307 P 74e71afd01564a22af6a83738c996a29: Deleting WAL directory at /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790257629041645-13911-0/minicluster-data/ts-1/wal/wals/48aabacfb90e45f7893abc7011e05307
I20260924 13:51:30.439895 32590 ts_tablet_manager.cc:1918] T 48aabacfb90e45f7893abc7011e05307 P 9e293413df4a44dfa8dc10b9c29a3564: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 13:51:30.439963 32590 log.cc:1196] T 48aabacfb90e45f7893abc7011e05307 P 9e293413df4a44dfa8dc10b9c29a3564: Deleting WAL directory at /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790257629041645-13911-0/minicluster-data/ts-2/wal/wals/48aabacfb90e45f7893abc7011e05307
I20260924 13:51:30.440222 32590 ts_tablet_manager.cc:1939] T 48aabacfb90e45f7893abc7011e05307 P 9e293413df4a44dfa8dc10b9c29a3564: Deleting consensus metadata
I20260924 13:51:30.440372 32591 ts_tablet_manager.cc:1939] T 48aabacfb90e45f7893abc7011e05307 P 74e71afd01564a22af6a83738c996a29: Deleting consensus metadata
I20260924 13:51:30.440799 32080 catalog_manager.cc:4841] TS 74e71afd01564a22af6a83738c996a29 (127.13.149.194:42405): tablet 48aabacfb90e45f7893abc7011e05307 (table test-table [id=30dbbaba685547b9bb495f8ca80b20aa]) successfully deleted
I20260924 13:51:30.440855 32079 catalog_manager.cc:4841] TS 9e293413df4a44dfa8dc10b9c29a3564 (127.13.149.195:42337): tablet 48aabacfb90e45f7893abc7011e05307 (table test-table [id=30dbbaba685547b9bb495f8ca80b20aa]) successfully deleted
W20260924 13:51:30.441797 32080 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.13.149.193:41815 (error 108)
I20260924 13:51:30.442263 32561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:30.442135 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.193:41815 (local address 127.0.0.1:49030)
0924 13:51:30.442189 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0924 13:51:30.442228 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.149.193:41815: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
I20260924 13:51:30.442343 13911 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJN9WRF/build/release/bin/kudu with pid 32289
W20260924 13:51:30.443697 32080 catalog_manager.cc:4584] TS 5519c1503daa4d9e91c1b9960b1eecf1 (127.13.149.193:41815): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 48aabacfb90e45f7893abc7011e05307: Network error: Client connection negotiation failed: client connection to 127.13.149.193:41815: connect: Connection refused (error 111)
I20260924 13:51:30.446930 13911 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJN9WRF/build/release/bin/kudu with pid 32423
I20260924 13:51:30.451078 13911 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJN9WRF/build/release/bin/kudu with pid 32061
2026-09-24T13:51:30Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (1829 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-09-24T13:51:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T13:51:30Z Disabled control of system clock
I20260924 13:51:30.477202 13911 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJN9WRF/build/release/bin/kudu
/tmp/dist-test-taskJN9WRF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790257629041645-13911-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790257629041645-13911-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790257629041645-13911-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790257629041645-13911-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.149.254:38865
--webserver_interface=127.13.149.254
--webserver_port=0
--builtin_ntp_servers=127.13.149.212:43623
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.149.254:38865
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260924 13:51:30.556010 32596 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 13:51:30.556249 32596 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 13:51:30.556268 32596 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 13:51:30.557430 32596 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 13:51:30.557466 32596 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 13:51:30.557478 32596 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 13:51:30.557488 32596 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 13:51:30.557499 32596 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 13:51:30.558789 32596 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.149.212:43623
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790257629041645-13911-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790257629041645-13911-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.149.254:38865
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.149.254:38865
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790257629041645-13911-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.149.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32596
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790257629041645-13911-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 12:29:37 UTC on ccycloud.quasar-iqzbxn.root.comops.site
build id 4670106
I20260924 13:51:30.558982 32596 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 13:51:30.559230 32596 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 13:51:30.563767 32602 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 13:51:30.563762 32601 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 13:51:30.563905 32596 server_base.cc:1034] running on GCE node
W20260924 13:51:30.563745 32604 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 13:51:30.564253 32596 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 13:51:30.564546 32596 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 13:51:30.565712 32596 hybrid_clock.cc:648] HybridClock initialized: now 1790257890565692 us; error 32 us; skew 500 ppm
I20260924 13:51:30.566901 32596 webserver.cc:460] Webserver started at http://127.13.149.254:41199/ using document root <none> and password file <none>
I20260924 13:51:30.567189 32596 fs_manager.cc:359] Metadata directory not provided
I20260924 13:51:30.567240 32596 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 13:51:30.567364 32596 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 13:51:30.568419 32596 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790257629041645-13911-0/minicluster-data/master-0/data/instance:
uuid: "218dbd383bd14b5bb0284d49e26cc2d1"
format_stamp: "Formatted at 2026-09-24 13:51:30 on dist-test-slave-sjl2"
server_key: "7b08643af14bdb200306254d77b1116d"
server_key_iv: "b15aa4e6afbad4a76f476446f7fedea1"
server_key_version: "encryptionkey@0"
I20260924 13:51:30.568742 32596 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790257629041645-13911-0/minicluster-data/master-0/wal/instance:
uuid: "218dbd383bd14b5bb0284d49e26cc2d1"
format_stamp: "Formatted at 2026-09-24 13:51:30 on dist-test-slave-sjl2"
server_key: "7b08643af14bdb200306254d77b1116d"
server_key_iv: "b15aa4e6afbad4a76f476446f7fedea1"
server_key_version: "encryptionkey@0"
I20260924 13:51:30.570029 32596 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260924 13:51:30.570807 32610 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 13:51:30.570968 32596 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 13:51:30.571058 32596 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790257629041645-13911-0/minicluster-data/master-0/data,/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790257629041645-13911-0/minicluster-data/master-0/wal
uuid: "218dbd383bd14b5bb0284d49e26cc2d1"
format_stamp: "Formatted at 2026-09-24 13:51:30 on dist-test-slave-sjl2"
server_key: "7b08643af14bdb200306254d77b1116d"
server_key_iv: "b15aa4e6afbad4a76f476446f7fedea1"
server_key_version: "encryptionkey@0"
I20260924 13:51:30.571170 32596 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790257629041645-13911-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790257629041645-13911-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790257629041645-13911-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 13:51:30.578971 32596 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 13:51:30.579248 32596 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 13:51:30.579367 32596 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 13:51:30.582949 32596 rpc_server.cc:307] RPC server started. Bound to: 127.13.149.254:38865
I20260924 13:51:30.582980 32662 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.149.254:38865 every 8 connection(s)
I20260924 13:51:30.583304 32596 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790257629041645-13911-0/minicluster-data/master-0/data/info.pb
I20260924 13:51:30.584192 32663 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 13:51:30.586486 32663 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 218dbd383bd14b5bb0284d49e26cc2d1: Bootstrap starting.
I20260924 13:51:30.587028 32663 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 218dbd383bd14b5bb0284d49e26cc2d1: Neither blocks nor log segments found. Creating new log.
I20260924 13:51:30.587286 32663 log.cc:824] T 00000000000000000000000000000000 P 218dbd383bd14b5bb0284d49e26cc2d1: Log is configured to *not* fsync() on all Append() calls
I20260924 13:51:30.587944 32663 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 218dbd383bd14b5bb0284d49e26cc2d1: No bootstrap required, opened a new log
I20260924 13:51:30.589589 32663 raft_consensus.cc:348] T 00000000000000000000000000000000 P 218dbd383bd14b5bb0284d49e26cc2d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "218dbd383bd14b5bb0284d49e26cc2d1" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 38865 } }
I20260924 13:51:30.589731 32663 raft_consensus.cc:374] T 00000000000000000000000000000000 P 218dbd383bd14b5bb0284d49e26cc2d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 13:51:30.589752 32663 raft_consensus.cc:729] T 00000000000000000000000000000000 P 218dbd383bd14b5bb0284d49e26cc2d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 218dbd383bd14b5bb0284d49e26cc2d1, State: Initialized, Role: FOLLOWER
I20260924 13:51:30.589879 32663 consensus_queue.cc:260] T 00000000000000000000000000000000 P 218dbd383bd14b5bb0284d49e26cc2d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "218dbd383bd14b5bb0284d49e26cc2d1" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 38865 } }
I20260924 13:51:30.589946 32663 raft_consensus.cc:388] T 00000000000000000000000000000000 P 218dbd383bd14b5bb0284d49e26cc2d1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 13:51:30.589969 32663 raft_consensus.cc:482] T 00000000000000000000000000000000 P 218dbd383bd14b5bb0284d49e26cc2d1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 13:51:30.590013 32663 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 218dbd383bd14b5bb0284d49e26cc2d1 [term 0 FOLLOWER]: Advancing to term 1
I20260924 13:51:30.590509 32663 raft_consensus.cc:504] T 00000000000000000000000000000000 P 218dbd383bd14b5bb0284d49e26cc2d1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "218dbd383bd14b5bb0284d49e26cc2d1" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 38865 } }
I20260924 13:51:30.590600 32663 leader_election.cc:302] T 00000000000000000000000000000000 P 218dbd383bd14b5bb0284d49e26cc2d1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 218dbd383bd14b5bb0284d49e26cc2d1; no voters:
I20260924 13:51:30.590756 32663 leader_election.cc:288] T 00000000000000000000000000000000 P 218dbd383bd14b5bb0284d49e26cc2d1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 13:51:30.590812 32666 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 218dbd383bd14b5bb0284d49e26cc2d1 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 13:51:30.590993 32663 sys_catalog.cc:564] T 00000000000000000000000000000000 P 218dbd383bd14b5bb0284d49e26cc2d1 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 13:51:30.591007 32666 raft_consensus.cc:686] T 00000000000000000000000000000000 P 218dbd383bd14b5bb0284d49e26cc2d1 [term 1 LEADER]: Becoming Leader. State: Replica: 218dbd383bd14b5bb0284d49e26cc2d1, State: Running, Role: LEADER
I20260924 13:51:30.591105 32666 consensus_queue.cc:237] T 00000000000000000000000000000000 P 218dbd383bd14b5bb0284d49e26cc2d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "218dbd383bd14b5bb0284d49e26cc2d1" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 38865 } }
I20260924 13:51:30.591375 13911 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJN9WRF/build/release/bin/kudu as pid 32596
I20260924 13:51:30.591466 13911 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790257629041645-13911-0/minicluster-data/master-0/wal/instance
I20260924 13:51:30.591615 32667 sys_catalog.cc:455] T 00000000000000000000000000000000 P 218dbd383bd14b5bb0284d49e26cc2d1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "218dbd383bd14b5bb0284d49e26cc2d1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "218dbd383bd14b5bb0284d49e26cc2d1" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 38865 } } }
I20260924 13:51:30.591686 13911 external_mini_cluster.cc:1428] Setting key 51224e10db61f10a292c0f675d9b3b47
I20260924 13:51:30.591712 32667 sys_catalog.cc:458] T 00000000000000000000000000000000 P 218dbd383bd14b5bb0284d49e26cc2d1 [sys.catalog]: This master's current role is: LEADER
I20260924 13:51:30.591883 32668 sys_catalog.cc:455] T 00000000000000000000000000000000 P 218dbd383bd14b5bb0284d49e26cc2d1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 218dbd383bd14b5bb0284d49e26cc2d1. Latest consensus state: current_term: 1 leader_uuid: "218dbd383bd14b5bb0284d49e26cc2d1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "218dbd383bd14b5bb0284d49e26cc2d1" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 38865 } } }
I20260924 13:51:30.591943 32668 sys_catalog.cc:458] T 00000000000000000000000000000000 P 218dbd383bd14b5bb0284d49e26cc2d1 [sys.catalog]: This master's current role is: LEADER
I20260924 13:51:30.592038 32675 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 13:51:30.592772 32675 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 13:51:30.594314 32675 catalog_manager.cc:1347] Generated new cluster ID: 2aa4afcac9f24af6a14e3514094656a9
I20260924 13:51:30.594367 32675 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 13:51:30.597065 32682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:30.592318 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44172 (local address 127.13.149.254:38865)
0924 13:51:30.592558 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0924 13:51:30.592568 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0924 13:51:30.592628 (+ 60us) server_negotiation.cc:407] Connection header received
0924 13:51:30.592767 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:51:30.592774 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:51:30.592843 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:51:30.593101 (+ 258us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:51:30.593246 (+ 145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:51:30.594411 (+ 1165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:51:30.595441 (+ 1030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:51:30.595832 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:51:30.595909 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:51:30.596462 (+ 553us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:51:30.596466 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:51:30.596466 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 13:51:30.596629 (+ 163us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:51:30.596661 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 13:51:30.596672 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:51:30.596719 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0924 13:51:30.596772 (+ 53us) server_negotiation.cc:299] Negotiation successful
0924 13:51:30.596838 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":122,"thread_start_us":75,"threads_started":1}
I20260924 13:51:30.611645 32675 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 13:51:30.612177 32675 catalog_manager.cc:1504] Loading token signing keys...
I20260924 13:51:30.616760 32675 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 218dbd383bd14b5bb0284d49e26cc2d1: Generated new TSK 0
I20260924 13:51:30.616966 32675 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260924 13:51:30.621369 13911 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJN9WRF/build/release/bin/kudu
/tmp/dist-test-taskJN9WRF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790257629041645-13911-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790257629041645-13911-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790257629041645-13911-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790257629041645-13911-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.149.193:0
--local_ip_for_outbound_sockets=127.13.149.193
--webserver_interface=127.13.149.193
--webserver_port=0
--tserver_master_addrs=127.13.149.254:38865
--builtin_ntp_servers=127.13.149.212:43623
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260924 13:51:30.704458 32687 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 13:51:30.704674 32687 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 13:51:30.704706 32687 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 13:51:30.705928 32687 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 13:51:30.705973 32687 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 13:51:30.706008 32687 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.149.193
I20260924 13:51:30.707399 32687 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.149.212:43623
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790257629041645-13911-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790257629041645-13911-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.149.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790257629041645-13911-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.149.193
--webserver_port=0
--tserver_master_addrs=127.13.149.254:38865
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32687
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.149.193
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790257629041645-13911-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 12:29:37 UTC on ccycloud.quasar-iqzbxn.root.comops.site
build id 4670106
I20260924 13:51:30.707634 32687 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 13:51:30.707927 32687 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 13:51:30.712810 32692 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 13:51:30.712783 32693 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 13:51:30.712783 32695 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 13:51:30.713002 32687 server_base.cc:1034] running on GCE node
I20260924 13:51:30.713276 32687 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 13:51:30.713523 32687 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 13:51:30.714694 32687 hybrid_clock.cc:648] HybridClock initialized: now 1790257890714678 us; error 33 us; skew 500 ppm
I20260924 13:51:30.715982 32687 webserver.cc:460] Webserver started at http://127.13.149.193:42911/ using document root <none> and password file <none>
I20260924 13:51:30.716224 32687 fs_manager.cc:359] Metadata directory not provided
I20260924 13:51:30.716272 32687 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 13:51:30.716390 32687 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 13:51:30.717563 32687 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790257629041645-13911-0/minicluster-data/ts-0/data/instance:
uuid: "5ad7c8617d4148dbb4e04a76cb48a7d2"
format_stamp: "Formatted at 2026-09-24 13:51:30 on dist-test-slave-sjl2"
server_key: "afb8711426b7526a0faf84fc7fb0814c"
server_key_iv: "a37326f7c7a67cd1985eae6ffc7fa996"
server_key_version: "encryptionkey@0"
I20260924 13:51:30.718073 32687 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790257629041645-13911-0/minicluster-data/ts-0/wal/instance:
uuid: "5ad7c8617d4148dbb4e04a76cb48a7d2"
format_stamp: "Formatted at 2026-09-24 13:51:30 on dist-test-slave-sjl2"
server_key: "afb8711426b7526a0faf84fc7fb0814c"
server_key_iv: "a37326f7c7a67cd1985eae6ffc7fa996"
server_key_version: "encryptionkey@0"
I20260924 13:51:30.719394 32687 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260924 13:51:30.720140 32701 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 13:51:30.720335 32687 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260924 13:51:30.720422 32687 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790257629041645-13911-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790257629041645-13911-0/minicluster-data/ts-0/wal
uuid: "5ad7c8617d4148dbb4e04a76cb48a7d2"
format_stamp: "Formatted at 2026-09-24 13:51:30 on dist-test-slave-sjl2"
server_key: "afb8711426b7526a0faf84fc7fb0814c"
server_key_iv: "a37326f7c7a67cd1985eae6ffc7fa996"
server_key_version: "encryptionkey@0"
I20260924 13:51:30.720502 32687 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790257629041645-13911-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790257629041645-13911-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790257629041645-13911-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 13:51:30.737068 32687 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 13:51:30.737396 32687 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 13:51:30.737527 32687 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 13:51:30.737756 32687 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 13:51:30.738078 32687 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 13:51:30.738121 32687 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 13:51:30.738152 32687 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 13:51:30.738175 32687 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 13:51:30.743527 32687 rpc_server.cc:307] RPC server started. Bound to: 127.13.149.193:46591
I20260924 13:51:30.743602 346 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.149.193:46591 every 8 connection(s)
I20260924 13:51:30.743884 32687 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790257629041645-13911-0/minicluster-data/ts-0/data/info.pb
I20260924 13:51:30.747936 13911 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJN9WRF/build/release/bin/kudu as pid 32687
I20260924 13:51:30.748054 13911 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790257629041645-13911-0/minicluster-data/ts-0/wal/instance
I20260924 13:51:30.748299 13911 external_mini_cluster.cc:1428] Setting key 85925b3e0c9d78402585aed6559aab66
I20260924 13:51:30.749298 32682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:30.744588 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.149.193:45671 (local address 127.13.149.254:38865)
0924 13:51:30.744657 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0924 13:51:30.744659 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:51:30.744888 (+ 229us) server_negotiation.cc:407] Connection header received
0924 13:51:30.745437 (+ 549us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:51:30.745438 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:51:30.745447 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:51:30.745498 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:51:30.746181 (+ 683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:51:30.746710 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:51:30.747683 (+ 973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:51:30.747915 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:51:30.747968 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:51:30.748941 (+ 973us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:51:30.748943 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:51:30.748943 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 13:51:30.748948 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:51:30.748965 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 13:51:30.748969 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:51:30.749020 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0924 13:51:30.749173 (+ 153us) server_negotiation.cc:299] Negotiation successful
0924 13:51:30.749214 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260924 13:51:30.749397 13911 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJN9WRF/build/release/bin/kudu
/tmp/dist-test-taskJN9WRF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790257629041645-13911-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790257629041645-13911-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790257629041645-13911-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790257629041645-13911-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.149.194:0
--local_ip_for_outbound_sockets=127.13.149.194
--webserver_interface=127.13.149.194
--webserver_port=0
--tserver_master_addrs=127.13.149.254:38865
--builtin_ntp_servers=127.13.149.212:43623
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260924 13:51:30.749442 349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:30.744581 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:38865 (local address 127.13.149.193:45671)
0924 13:51:30.744771 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0924 13:51:30.744805 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0924 13:51:30.745350 (+ 545us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:51:30.745565 (+ 215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:51:30.745569 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:51:30.745577 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0924 13:51:30.746080 (+ 503us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:51:30.746084 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:51:30.746792 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:51:30.746793 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:51:30.747586 (+ 793us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:51:30.747589 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:51:30.748021 (+ 432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:51:30.748022 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:51:30.748190 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:51:30.748761 (+ 571us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0924 13:51:30.748761 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0924 13:51:30.748799 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0924 13:51:30.748826 (+ 27us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 13:51:30.749030 (+ 204us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 13:51:30.749036 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 13:51:30.749039 (+ 3us) client_negotiation.cc:769] Sending connection context
0924 13:51:30.749158 (+ 119us) client_negotiation.cc:240] Negotiation successful
0924 13:51:30.749246 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":86,"threads_started":1}
I20260924 13:51:30.750869 347 heartbeater.cc:347] Connected to a master server at 127.13.149.254:38865
I20260924 13:51:30.750960 347 heartbeater.cc:464] Registering TS with master...
I20260924 13:51:30.751135 347 heartbeater.cc:511] Master 127.13.149.254:38865 requested a full tablet report, sending...
I20260924 13:51:30.751636 32627 ts_manager.cc:194] Registered new tserver with Master: 5ad7c8617d4148dbb4e04a76cb48a7d2 (127.13.149.193:46591)
I20260924 13:51:30.752497 32627 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.13.149.193:45671
W20260924 13:51:30.829865 350 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 13:51:30.830089 350 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 13:51:30.830111 350 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 13:51:30.831300 350 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 13:51:30.831343 350 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 13:51:30.831368 350 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.149.194
I20260924 13:51:30.832773 350 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.149.212:43623
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790257629041645-13911-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790257629041645-13911-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.149.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790257629041645-13911-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.149.194
--webserver_port=0
--tserver_master_addrs=127.13.149.254:38865
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.350
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.149.194
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790257629041645-13911-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 12:29:37 UTC on ccycloud.quasar-iqzbxn.root.comops.site
build id 4670106
I20260924 13:51:30.833037 350 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 13:51:30.833312 350 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 13:51:30.838356 355 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 13:51:30.838359 358 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 13:51:30.838407 350 server_base.cc:1034] running on GCE node
W20260924 13:51:30.838357 356 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 13:51:30.838788 350 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 13:51:30.839036 350 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 13:51:30.840178 350 hybrid_clock.cc:648] HybridClock initialized: now 1790257890840136 us; error 61 us; skew 500 ppm
I20260924 13:51:30.841459 350 webserver.cc:460] Webserver started at http://127.13.149.194:45567/ using document root <none> and password file <none>
I20260924 13:51:30.841701 350 fs_manager.cc:359] Metadata directory not provided
I20260924 13:51:30.841753 350 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 13:51:30.841881 350 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 13:51:30.842981 350 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790257629041645-13911-0/minicluster-data/ts-1/data/instance:
uuid: "7af24a9b7692461997f6e42c2b666edf"
format_stamp: "Formatted at 2026-09-24 13:51:30 on dist-test-slave-sjl2"
server_key: "a152196f7e5175471218dddb71241f45"
server_key_iv: "70ce89548d6a4a9a972ad4e95d7cacda"
server_key_version: "encryptionkey@0"
I20260924 13:51:30.843310 350 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790257629041645-13911-0/minicluster-data/ts-1/wal/instance:
uuid: "7af24a9b7692461997f6e42c2b666edf"
format_stamp: "Formatted at 2026-09-24 13:51:30 on dist-test-slave-sjl2"
server_key: "a152196f7e5175471218dddb71241f45"
server_key_iv: "70ce89548d6a4a9a972ad4e95d7cacda"
server_key_version: "encryptionkey@0"
I20260924 13:51:30.844559 350 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260924 13:51:30.845309 364 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 13:51:30.845484 350 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260924 13:51:30.845563 350 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790257629041645-13911-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790257629041645-13911-0/minicluster-data/ts-1/wal
uuid: "7af24a9b7692461997f6e42c2b666edf"
format_stamp: "Formatted at 2026-09-24 13:51:30 on dist-test-slave-sjl2"
server_key: "a152196f7e5175471218dddb71241f45"
server_key_iv: "70ce89548d6a4a9a972ad4e95d7cacda"
server_key_version: "encryptionkey@0"
I20260924 13:51:30.845634 350 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790257629041645-13911-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790257629041645-13911-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790257629041645-13911-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 13:51:30.861768 350 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 13:51:30.862092 350 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 13:51:30.862222 350 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 13:51:30.862447 350 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 13:51:30.862775 350 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 13:51:30.862824 350 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 13:51:30.862859 350 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 13:51:30.862881 350 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 13:51:30.868341 350 rpc_server.cc:307] RPC server started. Bound to: 127.13.149.194:34035
I20260924 13:51:30.868389 478 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.149.194:34035 every 8 connection(s)
I20260924 13:51:30.868713 350 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790257629041645-13911-0/minicluster-data/ts-1/data/info.pb
I20260924 13:51:30.874166 32682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:30.869446 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.149.194:47533 (local address 127.13.149.254:38865)
0924 13:51:30.869495 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0924 13:51:30.869497 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:51:30.869709 (+ 212us) server_negotiation.cc:407] Connection header received
0924 13:51:30.870211 (+ 502us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:51:30.870212 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:51:30.870226 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:51:30.870278 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:51:30.870953 (+ 675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:51:30.871436 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:51:30.872425 (+ 989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:51:30.872611 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:51:30.872678 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:51:30.873763 (+ 1085us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:51:30.873764 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:51:30.873765 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 13:51:30.873769 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:51:30.873784 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 13:51:30.873799 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:51:30.873840 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0924 13:51:30.874051 (+ 211us) server_negotiation.cc:299] Negotiation successful
0924 13:51:30.874087 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260924 13:51:30.874254 13911 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJN9WRF/build/release/bin/kudu as pid 350
I20260924 13:51:30.874344 13911 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790257629041645-13911-0/minicluster-data/ts-1/wal/instance
I20260924 13:51:30.874384 481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:30.869436 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:38865 (local address 127.13.149.194:47533)
0924 13:51:30.869600 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0924 13:51:30.869640 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0924 13:51:30.870122 (+ 482us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:51:30.870332 (+ 210us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:51:30.870335 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:51:30.870343 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0924 13:51:30.870879 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:51:30.870884 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:51:30.871517 (+ 633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:51:30.871518 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:51:30.872348 (+ 830us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:51:30.872351 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:51:30.872734 (+ 383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:51:30.872735 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:51:30.872883 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:51:30.873594 (+ 711us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0924 13:51:30.873595 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0924 13:51:30.873626 (+ 31us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0924 13:51:30.873653 (+ 27us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 13:51:30.873905 (+ 252us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 13:51:30.873914 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 13:51:30.873918 (+ 4us) client_negotiation.cc:769] Sending connection context
0924 13:51:30.873993 (+ 75us) client_negotiation.cc:240] Negotiation successful
0924 13:51:30.874107 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":75,"threads_started":1}
I20260924 13:51:30.874583 13911 external_mini_cluster.cc:1428] Setting key 8b783345547b5f6d3832f7f15b0e356f
I20260924 13:51:30.874845 479 heartbeater.cc:347] Connected to a master server at 127.13.149.254:38865
I20260924 13:51:30.874938 479 heartbeater.cc:464] Registering TS with master...
I20260924 13:51:30.875155 479 heartbeater.cc:511] Master 127.13.149.254:38865 requested a full tablet report, sending...
I20260924 13:51:30.875510 13911 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJN9WRF/build/release/bin/kudu
/tmp/dist-test-taskJN9WRF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790257629041645-13911-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790257629041645-13911-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790257629041645-13911-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790257629041645-13911-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.149.195:0
--local_ip_for_outbound_sockets=127.13.149.195
--webserver_interface=127.13.149.195
--webserver_port=0
--tserver_master_addrs=127.13.149.254:38865
--builtin_ntp_servers=127.13.149.212:43623
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260924 13:51:30.875511 32627 ts_manager.cc:194] Registered new tserver with Master: 7af24a9b7692461997f6e42c2b666edf (127.13.149.194:34035)
I20260924 13:51:30.875900 32627 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.13.149.194:47533
W20260924 13:51:30.956044 482 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 13:51:30.956261 482 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 13:51:30.956283 482 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 13:51:30.957554 482 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 13:51:30.957604 482 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 13:51:30.957640 482 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.149.195
I20260924 13:51:30.959321 482 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.149.212:43623
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790257629041645-13911-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790257629041645-13911-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.149.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790257629041645-13911-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.149.195
--webserver_port=0
--tserver_master_addrs=127.13.149.254:38865
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.482
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.149.195
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790257629041645-13911-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 12:29:37 UTC on ccycloud.quasar-iqzbxn.root.comops.site
build id 4670106
I20260924 13:51:30.959614 482 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 13:51:30.959933 482 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 13:51:30.965207 490 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 13:51:30.965230 488 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 13:51:30.965224 487 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 13:51:30.965354 482 server_base.cc:1034] running on GCE node
I20260924 13:51:30.965636 482 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 13:51:30.965881 482 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 13:51:30.967025 482 hybrid_clock.cc:648] HybridClock initialized: now 1790257890967012 us; error 30 us; skew 500 ppm
I20260924 13:51:30.968353 482 webserver.cc:460] Webserver started at http://127.13.149.195:36193/ using document root <none> and password file <none>
I20260924 13:51:30.968590 482 fs_manager.cc:359] Metadata directory not provided
I20260924 13:51:30.968634 482 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 13:51:30.968753 482 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 13:51:30.969905 482 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790257629041645-13911-0/minicluster-data/ts-2/data/instance:
uuid: "d3bb322529d249d2b6f051c60f7d3644"
format_stamp: "Formatted at 2026-09-24 13:51:30 on dist-test-slave-sjl2"
server_key: "433b9fef734a8ecba908f601bac00459"
server_key_iv: "d8d91b50cf57b38bcd72fee3c8a2886c"
server_key_version: "encryptionkey@0"
I20260924 13:51:30.970232 482 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790257629041645-13911-0/minicluster-data/ts-2/wal/instance:
uuid: "d3bb322529d249d2b6f051c60f7d3644"
format_stamp: "Formatted at 2026-09-24 13:51:30 on dist-test-slave-sjl2"
server_key: "433b9fef734a8ecba908f601bac00459"
server_key_iv: "d8d91b50cf57b38bcd72fee3c8a2886c"
server_key_version: "encryptionkey@0"
I20260924 13:51:30.971553 482 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260924 13:51:30.972270 496 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 13:51:30.972440 482 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 13:51:30.972522 482 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790257629041645-13911-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790257629041645-13911-0/minicluster-data/ts-2/wal
uuid: "d3bb322529d249d2b6f051c60f7d3644"
format_stamp: "Formatted at 2026-09-24 13:51:30 on dist-test-slave-sjl2"
server_key: "433b9fef734a8ecba908f601bac00459"
server_key_iv: "d8d91b50cf57b38bcd72fee3c8a2886c"
server_key_version: "encryptionkey@0"
I20260924 13:51:30.972587 482 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790257629041645-13911-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790257629041645-13911-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790257629041645-13911-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 13:51:30.990410 482 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 13:51:30.990715 482 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 13:51:30.990844 482 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 13:51:30.991067 482 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 13:51:30.991405 482 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 13:51:30.991447 482 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 13:51:30.991484 482 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 13:51:30.991508 482 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 13:51:30.996932 482 rpc_server.cc:307] RPC server started. Bound to: 127.13.149.195:46729
I20260924 13:51:30.996994 609 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.149.195:46729 every 8 connection(s)
I20260924 13:51:30.997359 482 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790257629041645-13911-0/minicluster-data/ts-2/data/info.pb
I20260924 13:51:31.000546 13911 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJN9WRF/build/release/bin/kudu as pid 482
I20260924 13:51:31.000643 13911 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790257629041645-13911-0/minicluster-data/ts-2/wal/instance
I20260924 13:51:31.000830 13911 external_mini_cluster.cc:1428] Setting key 6911b5c55960a4e18322dc2b90ea2e73
I20260924 13:51:31.002648 32682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:30.998085 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.149.195:40057 (local address 127.13.149.254:38865)
0924 13:51:30.998174 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0924 13:51:30.998176 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:51:30.998332 (+ 156us) server_negotiation.cc:407] Connection header received
0924 13:51:30.998813 (+ 481us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:51:30.998815 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:51:30.998824 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:51:30.998888 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:51:30.999607 (+ 719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:51:31.000145 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:51:31.001331 (+ 1186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:51:31.001517 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:51:31.001576 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:51:31.002393 (+ 817us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:51:31.002394 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:51:31.002394 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 13:51:31.002399 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:51:31.002414 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 13:51:31.002418 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:51:31.002460 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0924 13:51:31.002539 (+ 79us) server_negotiation.cc:299] Negotiation successful
0924 13:51:31.002567 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260924 13:51:31.002875 612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:30.998029 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:38865 (local address 127.13.149.195:40057)
0924 13:51:30.998220 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0924 13:51:30.998260 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0924 13:51:30.998736 (+ 476us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:51:30.998960 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:51:30.998963 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:51:30.998971 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0924 13:51:30.999485 (+ 514us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:51:30.999489 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:51:31.000254 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:51:31.000256 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:51:31.001214 (+ 958us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:51:31.001218 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:51:31.001592 (+ 374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:51:31.001593 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:51:31.001736 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:51:31.002262 (+ 526us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0924 13:51:31.002262 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0924 13:51:31.002292 (+ 30us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0924 13:51:31.002316 (+ 24us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 13:51:31.002467 (+ 151us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 13:51:31.002472 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 13:51:31.002475 (+ 3us) client_negotiation.cc:769] Sending connection context
0924 13:51:31.002538 (+ 63us) client_negotiation.cc:240] Negotiation successful
0924 13:51:31.002643 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":73,"threads_started":1}
I20260924 13:51:31.003394 610 heartbeater.cc:347] Connected to a master server at 127.13.149.254:38865
I20260924 13:51:31.003482 610 heartbeater.cc:464] Registering TS with master...
I20260924 13:51:31.003664 610 heartbeater.cc:511] Master 127.13.149.254:38865 requested a full tablet report, sending...
I20260924 13:51:31.004079 32627 ts_manager.cc:194] Registered new tserver with Master: d3bb322529d249d2b6f051c60f7d3644 (127.13.149.195:46729)
I20260924 13:51:31.004441 32627 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.13.149.195:40057
I20260924 13:51:31.012666 13911 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260924 13:51:31.017162 32682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:31.014115 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44180 (local address 127.13.149.254:38865)
0924 13:51:31.014202 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0924 13:51:31.014205 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 13:51:31.014256 (+ 51us) server_negotiation.cc:407] Connection header received
0924 13:51:31.014353 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:51:31.014354 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:51:31.014364 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:51:31.014413 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:51:31.014577 (+ 164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:51:31.015074 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:51:31.015843 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:51:31.016041 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:51:31.016098 (+ 57us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:51:31.016839 (+ 741us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:51:31.016840 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:51:31.016841 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 13:51:31.016846 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:51:31.016862 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 13:51:31.016867 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:51:31.016961 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0924 13:51:31.017056 (+ 95us) server_negotiation.cc:299] Negotiation successful
0924 13:51:31.017067 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260924 13:51:31.018806 32627 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44180:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 13:51:31.020360 32627 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 13:51:31.026750 620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:31.022456 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.195:46729 (local address 127.0.0.1:44948)
0924 13:51:31.022836 (+ 380us) negotiation.cc:107] Waiting for socket to connect
0924 13:51:31.022844 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0924 13:51:31.022881 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:51:31.023309 (+ 428us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:51:31.023310 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:51:31.023312 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 13:51:31.023356 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:51:31.023358 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:51:31.024549 (+ 1191us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:51:31.024550 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:51:31.025582 (+ 1032us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:51:31.025585 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:51:31.026435 (+ 850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:51:31.026436 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:51:31.026509 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:51:31.026522 (+ 13us) client_negotiation.cc:769] Sending connection context
0924 13:51:31.026603 (+ 81us) client_negotiation.cc:240] Negotiation successful
0924 13:51:31.026662 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":366,"mutex_wait_us":2,"thread_start_us":105,"threads_started":1}
I20260924 13:51:31.026763 622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:31.022696 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44948 (local address 127.13.149.195:46729)
0924 13:51:31.023100 (+ 404us) server_negotiation.cc:206] Beginning negotiation
0924 13:51:31.023105 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 13:51:31.023114 (+ 9us) server_negotiation.cc:407] Connection header received
0924 13:51:31.023154 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:51:31.023156 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:51:31.023170 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:51:31.023242 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 13:51:31.023429 (+ 187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:51:31.024493 (+ 1064us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:51:31.025656 (+ 1163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:51:31.026331 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:51:31.026412 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:51:31.026506 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0924 13:51:31.026619 (+ 113us) server_negotiation.cc:299] Negotiation successful
0924 13:51:31.026661 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":368,"thread_start_us":73,"threads_started":1}
I20260924 13:51:31.026850 619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:31.022448 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.194:34035 (local address 127.0.0.1:38004)
0924 13:51:31.022609 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0924 13:51:31.022653 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0924 13:51:31.022724 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:51:31.022985 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:51:31.022988 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:51:31.022991 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 13:51:31.023068 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:51:31.023070 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:51:31.024467 (+ 1397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:51:31.024468 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:51:31.025778 (+ 1310us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:51:31.025781 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:51:31.026650 (+ 869us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:51:31.026651 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:51:31.026701 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:51:31.026705 (+ 4us) client_negotiation.cc:769] Sending connection context
0924 13:51:31.026753 (+ 48us) client_negotiation.cc:240] Negotiation successful
0924 13:51:31.026789 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"mutex_wait_us":92,"thread_start_us":60,"threads_started":1}
I20260924 13:51:31.027046 621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:31.022608 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38004 (local address 127.13.149.194:34035)
0924 13:51:31.022787 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0924 13:51:31.022792 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 13:51:31.022803 (+ 11us) server_negotiation.cc:407] Connection header received
0924 13:51:31.022869 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:51:31.022872 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:51:31.022887 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:51:31.022981 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 13:51:31.023141 (+ 160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:51:31.024389 (+ 1248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:51:31.025865 (+ 1476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:51:31.026554 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:51:31.026627 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:51:31.026702 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0924 13:51:31.026916 (+ 214us) server_negotiation.cc:299] Negotiation successful
0924 13:51:31.026964 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":133,"thread_start_us":93,"threads_started":1}
I20260924 13:51:31.027674 544 tablet_service.cc:1467] Processing CreateTablet for tablet 75eddaf7f0044935ab31d8d4ca836665 (DEFAULT_TABLE table=test-table [id=fe7831cb87c94a86aae4d6c84df125c6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 13:51:31.027798 413 tablet_service.cc:1467] Processing CreateTablet for tablet 75eddaf7f0044935ab31d8d4ca836665 (DEFAULT_TABLE table=test-table [id=fe7831cb87c94a86aae4d6c84df125c6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 13:51:31.028107 618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:31.022456 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.193:46591 (local address 127.0.0.1:36142)
0924 13:51:31.022828 (+ 372us) negotiation.cc:107] Waiting for socket to connect
0924 13:51:31.022837 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0924 13:51:31.022881 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:51:31.023303 (+ 422us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:51:31.023305 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:51:31.023307 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 13:51:31.023348 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:51:31.023349 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:51:31.024355 (+ 1006us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:51:31.024356 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:51:31.025777 (+ 1421us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:51:31.025781 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:51:31.027892 (+ 2111us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:51:31.027893 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:51:31.027957 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:51:31.027962 (+ 5us) client_negotiation.cc:769] Sending connection context
0924 13:51:31.027993 (+ 31us) client_negotiation.cc:240] Negotiation successful
0924 13:51:31.028037 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":327,"mutex_wait_us":59,"thread_start_us":90,"threads_started":1}
I20260924 13:51:31.028038 544 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 75eddaf7f0044935ab31d8d4ca836665. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 13:51:31.028163 413 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 75eddaf7f0044935ab31d8d4ca836665. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 13:51:31.028566 623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:31.022893 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36142 (local address 127.13.149.193:46591)
0924 13:51:31.023054 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0924 13:51:31.023058 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 13:51:31.023067 (+ 9us) server_negotiation.cc:407] Connection header received
0924 13:51:31.023105 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:51:31.023107 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:51:31.023122 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:51:31.023190 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 13:51:31.023426 (+ 236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:51:31.024264 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:51:31.026747 (+ 2483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:51:31.027787 (+ 1040us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:51:31.028312 (+ 525us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:51:31.028418 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0924 13:51:31.028445 (+ 27us) server_negotiation.cc:299] Negotiation successful
0924 13:51:31.028488 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":121,"thread_start_us":83,"threads_started":1}
I20260924 13:51:31.029261 32749 tablet_service.cc:1467] Processing CreateTablet for tablet 75eddaf7f0044935ab31d8d4ca836665 (DEFAULT_TABLE table=test-table [id=fe7831cb87c94a86aae4d6c84df125c6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 13:51:31.029633 32749 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 75eddaf7f0044935ab31d8d4ca836665. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 13:51:31.030848 624 tablet_bootstrap.cc:492] T 75eddaf7f0044935ab31d8d4ca836665 P 7af24a9b7692461997f6e42c2b666edf: Bootstrap starting.
I20260924 13:51:31.031523 624 tablet_bootstrap.cc:654] T 75eddaf7f0044935ab31d8d4ca836665 P 7af24a9b7692461997f6e42c2b666edf: Neither blocks nor log segments found. Creating new log.
I20260924 13:51:31.031778 624 log.cc:824] T 75eddaf7f0044935ab31d8d4ca836665 P 7af24a9b7692461997f6e42c2b666edf: Log is configured to *not* fsync() on all Append() calls
I20260924 13:51:31.032470 624 tablet_bootstrap.cc:492] T 75eddaf7f0044935ab31d8d4ca836665 P 7af24a9b7692461997f6e42c2b666edf: No bootstrap required, opened a new log
I20260924 13:51:31.032514 624 ts_tablet_manager.cc:1397] T 75eddaf7f0044935ab31d8d4ca836665 P 7af24a9b7692461997f6e42c2b666edf: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260924 13:51:31.032747 627 tablet_bootstrap.cc:492] T 75eddaf7f0044935ab31d8d4ca836665 P 5ad7c8617d4148dbb4e04a76cb48a7d2: Bootstrap starting.
I20260924 13:51:31.033268 625 tablet_bootstrap.cc:492] T 75eddaf7f0044935ab31d8d4ca836665 P d3bb322529d249d2b6f051c60f7d3644: Bootstrap starting.
I20260924 13:51:31.033423 627 tablet_bootstrap.cc:654] T 75eddaf7f0044935ab31d8d4ca836665 P 5ad7c8617d4148dbb4e04a76cb48a7d2: Neither blocks nor log segments found. Creating new log.
I20260924 13:51:31.033744 627 log.cc:824] T 75eddaf7f0044935ab31d8d4ca836665 P 5ad7c8617d4148dbb4e04a76cb48a7d2: Log is configured to *not* fsync() on all Append() calls
I20260924 13:51:31.033926 625 tablet_bootstrap.cc:654] T 75eddaf7f0044935ab31d8d4ca836665 P d3bb322529d249d2b6f051c60f7d3644: Neither blocks nor log segments found. Creating new log.
I20260924 13:51:31.034238 625 log.cc:824] T 75eddaf7f0044935ab31d8d4ca836665 P d3bb322529d249d2b6f051c60f7d3644: Log is configured to *not* fsync() on all Append() calls
I20260924 13:51:31.034487 627 tablet_bootstrap.cc:492] T 75eddaf7f0044935ab31d8d4ca836665 P 5ad7c8617d4148dbb4e04a76cb48a7d2: No bootstrap required, opened a new log
I20260924 13:51:31.034559 627 ts_tablet_manager.cc:1397] T 75eddaf7f0044935ab31d8d4ca836665 P 5ad7c8617d4148dbb4e04a76cb48a7d2: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260924 13:51:31.034816 624 raft_consensus.cc:348] T 75eddaf7f0044935ab31d8d4ca836665 P 7af24a9b7692461997f6e42c2b666edf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7af24a9b7692461997f6e42c2b666edf" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 34035 } } peers { permanent_uuid: "5ad7c8617d4148dbb4e04a76cb48a7d2" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 46591 } } peers { permanent_uuid: "d3bb322529d249d2b6f051c60f7d3644" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 46729 } }
I20260924 13:51:31.034960 624 raft_consensus.cc:374] T 75eddaf7f0044935ab31d8d4ca836665 P 7af24a9b7692461997f6e42c2b666edf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 13:51:31.034992 624 raft_consensus.cc:729] T 75eddaf7f0044935ab31d8d4ca836665 P 7af24a9b7692461997f6e42c2b666edf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7af24a9b7692461997f6e42c2b666edf, State: Initialized, Role: FOLLOWER
I20260924 13:51:31.035130 624 consensus_queue.cc:260] T 75eddaf7f0044935ab31d8d4ca836665 P 7af24a9b7692461997f6e42c2b666edf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7af24a9b7692461997f6e42c2b666edf" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 34035 } } peers { permanent_uuid: "5ad7c8617d4148dbb4e04a76cb48a7d2" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 46591 } } peers { permanent_uuid: "d3bb322529d249d2b6f051c60f7d3644" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 46729 } }
I20260924 13:51:31.035178 625 tablet_bootstrap.cc:492] T 75eddaf7f0044935ab31d8d4ca836665 P d3bb322529d249d2b6f051c60f7d3644: No bootstrap required, opened a new log
I20260924 13:51:31.035244 625 ts_tablet_manager.cc:1397] T 75eddaf7f0044935ab31d8d4ca836665 P d3bb322529d249d2b6f051c60f7d3644: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260924 13:51:31.035396 624 ts_tablet_manager.cc:1428] T 75eddaf7f0044935ab31d8d4ca836665 P 7af24a9b7692461997f6e42c2b666edf: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260924 13:51:31.035526 479 heartbeater.cc:503] Master 127.13.149.254:38865 was elected leader, sending a full tablet report...
I20260924 13:51:31.037175 627 raft_consensus.cc:348] T 75eddaf7f0044935ab31d8d4ca836665 P 5ad7c8617d4148dbb4e04a76cb48a7d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7af24a9b7692461997f6e42c2b666edf" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 34035 } } peers { permanent_uuid: "5ad7c8617d4148dbb4e04a76cb48a7d2" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 46591 } } peers { permanent_uuid: "d3bb322529d249d2b6f051c60f7d3644" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 46729 } }
I20260924 13:51:31.037366 627 raft_consensus.cc:374] T 75eddaf7f0044935ab31d8d4ca836665 P 5ad7c8617d4148dbb4e04a76cb48a7d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 13:51:31.037415 627 raft_consensus.cc:729] T 75eddaf7f0044935ab31d8d4ca836665 P 5ad7c8617d4148dbb4e04a76cb48a7d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5ad7c8617d4148dbb4e04a76cb48a7d2, State: Initialized, Role: FOLLOWER
I20260924 13:51:31.037545 627 consensus_queue.cc:260] T 75eddaf7f0044935ab31d8d4ca836665 P 5ad7c8617d4148dbb4e04a76cb48a7d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7af24a9b7692461997f6e42c2b666edf" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 34035 } } peers { permanent_uuid: "5ad7c8617d4148dbb4e04a76cb48a7d2" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 46591 } } peers { permanent_uuid: "d3bb322529d249d2b6f051c60f7d3644" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 46729 } }
I20260924 13:51:31.037850 347 heartbeater.cc:503] Master 127.13.149.254:38865 was elected leader, sending a full tablet report...
I20260924 13:51:31.037839 625 raft_consensus.cc:348] T 75eddaf7f0044935ab31d8d4ca836665 P d3bb322529d249d2b6f051c60f7d3644 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7af24a9b7692461997f6e42c2b666edf" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 34035 } } peers { permanent_uuid: "5ad7c8617d4148dbb4e04a76cb48a7d2" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 46591 } } peers { permanent_uuid: "d3bb322529d249d2b6f051c60f7d3644" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 46729 } }
I20260924 13:51:31.038010 625 raft_consensus.cc:374] T 75eddaf7f0044935ab31d8d4ca836665 P d3bb322529d249d2b6f051c60f7d3644 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 13:51:31.038050 625 raft_consensus.cc:729] T 75eddaf7f0044935ab31d8d4ca836665 P d3bb322529d249d2b6f051c60f7d3644 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d3bb322529d249d2b6f051c60f7d3644, State: Initialized, Role: FOLLOWER
I20260924 13:51:31.038170 627 ts_tablet_manager.cc:1428] T 75eddaf7f0044935ab31d8d4ca836665 P 5ad7c8617d4148dbb4e04a76cb48a7d2: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260924 13:51:31.038158 625 consensus_queue.cc:260] T 75eddaf7f0044935ab31d8d4ca836665 P d3bb322529d249d2b6f051c60f7d3644 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7af24a9b7692461997f6e42c2b666edf" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 34035 } } peers { permanent_uuid: "5ad7c8617d4148dbb4e04a76cb48a7d2" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 46591 } } peers { permanent_uuid: "d3bb322529d249d2b6f051c60f7d3644" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 46729 } }
I20260924 13:51:31.038409 625 ts_tablet_manager.cc:1428] T 75eddaf7f0044935ab31d8d4ca836665 P d3bb322529d249d2b6f051c60f7d3644: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260924 13:51:31.038439 610 heartbeater.cc:503] Master 127.13.149.254:38865 was elected leader, sending a full tablet report...
I20260924 13:51:31.039918 630 raft_consensus.cc:482] T 75eddaf7f0044935ab31d8d4ca836665 P 7af24a9b7692461997f6e42c2b666edf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 13:51:31.040002 630 raft_consensus.cc:504] T 75eddaf7f0044935ab31d8d4ca836665 P 7af24a9b7692461997f6e42c2b666edf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7af24a9b7692461997f6e42c2b666edf" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 34035 } } peers { permanent_uuid: "5ad7c8617d4148dbb4e04a76cb48a7d2" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 46591 } } peers { permanent_uuid: "d3bb322529d249d2b6f051c60f7d3644" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 46729 } }
I20260924 13:51:31.040334 630 leader_election.cc:288] T 75eddaf7f0044935ab31d8d4ca836665 P 7af24a9b7692461997f6e42c2b666edf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5ad7c8617d4148dbb4e04a76cb48a7d2 (127.13.149.193:46591), d3bb322529d249d2b6f051c60f7d3644 (127.13.149.195:46729)
I20260924 13:51:31.043298 623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:31.040630 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.149.194:55091 (local address 127.13.149.193:46591)
0924 13:51:31.040697 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0924 13:51:31.040699 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:51:31.040708 (+ 9us) server_negotiation.cc:407] Connection header received
0924 13:51:31.040730 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:51:31.040731 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:51:31.040740 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:51:31.040791 (+ 51us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 13:51:31.040968 (+ 177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:51:31.041438 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:51:31.042541 (+ 1103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:51:31.042960 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:51:31.043048 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:51:31.043056 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0924 13:51:31.043194 (+ 138us) server_negotiation.cc:299] Negotiation successful
0924 13:51:31.043233 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260924 13:51:31.043335 481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:31.040476 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.193:46591 (local address 127.13.149.194:55091)
0924 13:51:31.040528 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0924 13:51:31.040538 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 13:51:31.040570 (+ 32us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:51:31.040804 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:51:31.040805 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:51:31.040806 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 13:51:31.040884 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:51:31.040886 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:51:31.041516 (+ 630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:51:31.041517 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:51:31.042463 (+ 946us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:51:31.042465 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:51:31.043056 (+ 591us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:51:31.043057 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:51:31.043134 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:51:31.043137 (+ 3us) client_negotiation.cc:769] Sending connection context
0924 13:51:31.043180 (+ 43us) client_negotiation.cc:240] Negotiation successful
0924 13:51:31.043226 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37,"mutex_wait_us":26}
I20260924 13:51:31.043329 622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:31.040674 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.149.194:48219 (local address 127.13.149.195:46729)
0924 13:51:31.040731 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0924 13:51:31.040733 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:51:31.040740 (+ 7us) server_negotiation.cc:407] Connection header received
0924 13:51:31.040763 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:51:31.040764 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:51:31.040772 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:51:31.040821 (+ 49us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 13:51:31.040983 (+ 162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:51:31.041450 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:51:31.042511 (+ 1061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:51:31.042971 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:51:31.043034 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:51:31.043043 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0924 13:51:31.043206 (+ 163us) server_negotiation.cc:299] Negotiation successful
0924 13:51:31.043240 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260924 13:51:31.043350 633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:31.040488 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.195:46729 (local address 127.13.149.194:48219)
0924 13:51:31.040646 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0924 13:51:31.040654 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0924 13:51:31.040693 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:51:31.040837 (+ 144us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:51:31.040838 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:51:31.040840 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 13:51:31.040904 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:51:31.040905 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:51:31.041522 (+ 617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:51:31.041523 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:51:31.042424 (+ 901us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:51:31.042427 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:51:31.043048 (+ 621us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:51:31.043049 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:51:31.043143 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:51:31.043146 (+ 3us) client_negotiation.cc:769] Sending connection context
0924 13:51:31.043194 (+ 48us) client_negotiation.cc:240] Negotiation successful
0924 13:51:31.043232 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":95,"threads_started":1}
I20260924 13:51:31.043735 564 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "75eddaf7f0044935ab31d8d4ca836665" candidate_uuid: "7af24a9b7692461997f6e42c2b666edf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3bb322529d249d2b6f051c60f7d3644" is_pre_election: true
I20260924 13:51:31.043821 301 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "75eddaf7f0044935ab31d8d4ca836665" candidate_uuid: "7af24a9b7692461997f6e42c2b666edf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ad7c8617d4148dbb4e04a76cb48a7d2" is_pre_election: true
I20260924 13:51:31.043888 564 raft_consensus.cc:2454] T 75eddaf7f0044935ab31d8d4ca836665 P d3bb322529d249d2b6f051c60f7d3644 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7af24a9b7692461997f6e42c2b666edf in term 0.
I20260924 13:51:31.043946 301 raft_consensus.cc:2454] T 75eddaf7f0044935ab31d8d4ca836665 P 5ad7c8617d4148dbb4e04a76cb48a7d2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7af24a9b7692461997f6e42c2b666edf in term 0.
I20260924 13:51:31.044095 366 leader_election.cc:302] T 75eddaf7f0044935ab31d8d4ca836665 P 7af24a9b7692461997f6e42c2b666edf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7af24a9b7692461997f6e42c2b666edf, d3bb322529d249d2b6f051c60f7d3644; no voters:
I20260924 13:51:31.044229 630 raft_consensus.cc:2781] T 75eddaf7f0044935ab31d8d4ca836665 P 7af24a9b7692461997f6e42c2b666edf [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 13:51:31.044292 630 raft_consensus.cc:482] T 75eddaf7f0044935ab31d8d4ca836665 P 7af24a9b7692461997f6e42c2b666edf [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 13:51:31.044308 630 raft_consensus.cc:3037] T 75eddaf7f0044935ab31d8d4ca836665 P 7af24a9b7692461997f6e42c2b666edf [term 0 FOLLOWER]: Advancing to term 1
I20260924 13:51:31.045054 630 raft_consensus.cc:504] T 75eddaf7f0044935ab31d8d4ca836665 P 7af24a9b7692461997f6e42c2b666edf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7af24a9b7692461997f6e42c2b666edf" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 34035 } } peers { permanent_uuid: "5ad7c8617d4148dbb4e04a76cb48a7d2" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 46591 } } peers { permanent_uuid: "d3bb322529d249d2b6f051c60f7d3644" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 46729 } }
I20260924 13:51:31.045189 630 leader_election.cc:288] T 75eddaf7f0044935ab31d8d4ca836665 P 7af24a9b7692461997f6e42c2b666edf [CANDIDATE]: Term 1 election: Requested vote from peers 5ad7c8617d4148dbb4e04a76cb48a7d2 (127.13.149.193:46591), d3bb322529d249d2b6f051c60f7d3644 (127.13.149.195:46729)
I20260924 13:51:31.045336 301 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "75eddaf7f0044935ab31d8d4ca836665" candidate_uuid: "7af24a9b7692461997f6e42c2b666edf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ad7c8617d4148dbb4e04a76cb48a7d2"
I20260924 13:51:31.045375 564 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "75eddaf7f0044935ab31d8d4ca836665" candidate_uuid: "7af24a9b7692461997f6e42c2b666edf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3bb322529d249d2b6f051c60f7d3644"
I20260924 13:51:31.045440 564 raft_consensus.cc:3037] T 75eddaf7f0044935ab31d8d4ca836665 P d3bb322529d249d2b6f051c60f7d3644 [term 0 FOLLOWER]: Advancing to term 1
I20260924 13:51:31.045419 301 raft_consensus.cc:3037] T 75eddaf7f0044935ab31d8d4ca836665 P 5ad7c8617d4148dbb4e04a76cb48a7d2 [term 0 FOLLOWER]: Advancing to term 1
I20260924 13:51:31.046110 564 raft_consensus.cc:2454] T 75eddaf7f0044935ab31d8d4ca836665 P d3bb322529d249d2b6f051c60f7d3644 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7af24a9b7692461997f6e42c2b666edf in term 1.
I20260924 13:51:31.046190 301 raft_consensus.cc:2454] T 75eddaf7f0044935ab31d8d4ca836665 P 5ad7c8617d4148dbb4e04a76cb48a7d2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7af24a9b7692461997f6e42c2b666edf in term 1.
I20260924 13:51:31.046273 366 leader_election.cc:302] T 75eddaf7f0044935ab31d8d4ca836665 P 7af24a9b7692461997f6e42c2b666edf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7af24a9b7692461997f6e42c2b666edf, d3bb322529d249d2b6f051c60f7d3644; no voters:
I20260924 13:51:31.046378 630 raft_consensus.cc:2781] T 75eddaf7f0044935ab31d8d4ca836665 P 7af24a9b7692461997f6e42c2b666edf [term 1 FOLLOWER]: Leader election won for term 1
I20260924 13:51:31.046550 630 raft_consensus.cc:686] T 75eddaf7f0044935ab31d8d4ca836665 P 7af24a9b7692461997f6e42c2b666edf [term 1 LEADER]: Becoming Leader. State: Replica: 7af24a9b7692461997f6e42c2b666edf, State: Running, Role: LEADER
I20260924 13:51:31.046636 630 consensus_queue.cc:237] T 75eddaf7f0044935ab31d8d4ca836665 P 7af24a9b7692461997f6e42c2b666edf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7af24a9b7692461997f6e42c2b666edf" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 34035 } } peers { permanent_uuid: "5ad7c8617d4148dbb4e04a76cb48a7d2" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 46591 } } peers { permanent_uuid: "d3bb322529d249d2b6f051c60f7d3644" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 46729 } }
I20260924 13:51:31.047398 32626 catalog_manager.cc:5495] T 75eddaf7f0044935ab31d8d4ca836665 P 7af24a9b7692461997f6e42c2b666edf reported cstate change: term changed from 0 to 1, leader changed from <none> to 7af24a9b7692461997f6e42c2b666edf (127.13.149.194). New cstate: current_term: 1 leader_uuid: "7af24a9b7692461997f6e42c2b666edf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7af24a9b7692461997f6e42c2b666edf" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 34035 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5ad7c8617d4148dbb4e04a76cb48a7d2" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 46591 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d3bb322529d249d2b6f051c60f7d3644" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 46729 } health_report { overall_health: UNKNOWN } } }
I20260924 13:51:31.058097 621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:31.055836 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38020 (local address 127.13.149.194:34035)
0924 13:51:31.055893 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0924 13:51:31.055895 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:51:31.055903 (+ 8us) server_negotiation.cc:407] Connection header received
0924 13:51:31.055938 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:51:31.055940 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:51:31.055958 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:51:31.056014 (+ 56us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 13:51:31.056131 (+ 117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:51:31.056638 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:51:31.057408 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:51:31.057592 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:51:31.057644 (+ 52us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:51:31.057780 (+ 136us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 13:51:31.057870 (+ 90us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 13:51:31.057953 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0924 13:51:31.057999 (+ 46us) server_negotiation.cc:299] Negotiation successful
0924 13:51:31.058008 (+ 9us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260924 13:51:31.059844 564 raft_consensus.cc:1261] T 75eddaf7f0044935ab31d8d4ca836665 P d3bb322529d249d2b6f051c60f7d3644 [term 1 FOLLOWER]: Refusing update from remote peer 7af24a9b7692461997f6e42c2b666edf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 13:51:31.059846 301 raft_consensus.cc:1261] T 75eddaf7f0044935ab31d8d4ca836665 P 5ad7c8617d4148dbb4e04a76cb48a7d2 [term 1 FOLLOWER]: Refusing update from remote peer 7af24a9b7692461997f6e42c2b666edf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 13:51:31.060189 634 consensus_queue.cc:1035] T 75eddaf7f0044935ab31d8d4ca836665 P 7af24a9b7692461997f6e42c2b666edf [LEADER]: Connected to new peer: Peer: permanent_uuid: "5ad7c8617d4148dbb4e04a76cb48a7d2" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 46591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 13:51:31.060307 630 consensus_queue.cc:1035] T 75eddaf7f0044935ab31d8d4ca836665 P 7af24a9b7692461997f6e42c2b666edf [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3bb322529d249d2b6f051c60f7d3644" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 46729 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 13:51:31.062597 637 mvcc.cc:205] Tried to move back new op lower bound from 7332896321778614272 to 7332896321727565824. Current Snapshot: MvccSnapshot[applied={T|T < 7332896321778614272}]
I20260924 13:51:31.063448 639 mvcc.cc:205] Tried to move back new op lower bound from 7332896321778614272 to 7332896321727565824. Current Snapshot: MvccSnapshot[applied={T|T < 7332896321778614272 or (T in {7332896321778614272})}]
I20260924 13:51:31.063575 638 mvcc.cc:205] Tried to move back new op lower bound from 7332896321778614272 to 7332896321727565824. Current Snapshot: MvccSnapshot[applied={T|T < 7332896321778614272}]
I20260924 13:51:31.064900 32627 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44180:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 13:51:31.065039 32627 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44180:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 13:51:31.065811 32627 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 218dbd383bd14b5bb0284d49e26cc2d1: Sending DeleteTablet for 3 replicas of tablet 75eddaf7f0044935ab31d8d4ca836665
I20260924 13:51:31.066154 544 tablet_service.cc:1514] Processing DeleteTablet for tablet 75eddaf7f0044935ab31d8d4ca836665 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 13:51:31 UTC) from {username='slave'} at 127.0.0.1:44948
I20260924 13:51:31.066175 32749 tablet_service.cc:1514] Processing DeleteTablet for tablet 75eddaf7f0044935ab31d8d4ca836665 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 13:51:31 UTC) from {username='slave'} at 127.0.0.1:36142
I20260924 13:51:31.066416 647 tablet_replica.cc:330] T 75eddaf7f0044935ab31d8d4ca836665 P d3bb322529d249d2b6f051c60f7d3644: stopping tablet replica
I20260924 13:51:31.066530 647 raft_consensus.cc:2229] T 75eddaf7f0044935ab31d8d4ca836665 P d3bb322529d249d2b6f051c60f7d3644 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 13:51:31.066555 413 tablet_service.cc:1514] Processing DeleteTablet for tablet 75eddaf7f0044935ab31d8d4ca836665 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 13:51:31 UTC) from {username='slave'} at 127.0.0.1:38004
I20260924 13:51:31.066651 647 raft_consensus.cc:2258] T 75eddaf7f0044935ab31d8d4ca836665 P d3bb322529d249d2b6f051c60f7d3644 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 13:51:31.066783 649 tablet_replica.cc:330] T 75eddaf7f0044935ab31d8d4ca836665 P 7af24a9b7692461997f6e42c2b666edf: stopping tablet replica
I20260924 13:51:31.066886 649 raft_consensus.cc:2229] T 75eddaf7f0044935ab31d8d4ca836665 P 7af24a9b7692461997f6e42c2b666edf [term 1 LEADER]: Raft consensus shutting down.
I20260924 13:51:31.066987 649 raft_consensus.cc:2258] T 75eddaf7f0044935ab31d8d4ca836665 P 7af24a9b7692461997f6e42c2b666edf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 13:51:31.067052 13911 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJN9WRF/build/release/bin/kudu with pid 32687
I20260924 13:51:31.068037 649 ts_tablet_manager.cc:1905] T 75eddaf7f0044935ab31d8d4ca836665 P 7af24a9b7692461997f6e42c2b666edf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 13:51:31.069849 649 ts_tablet_manager.cc:1918] T 75eddaf7f0044935ab31d8d4ca836665 P 7af24a9b7692461997f6e42c2b666edf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 13:51:31.069926 649 log.cc:1196] T 75eddaf7f0044935ab31d8d4ca836665 P 7af24a9b7692461997f6e42c2b666edf: Deleting WAL directory at /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790257629041645-13911-0/minicluster-data/ts-1/wal/wals/75eddaf7f0044935ab31d8d4ca836665
I20260924 13:51:31.070180 649 ts_tablet_manager.cc:1939] T 75eddaf7f0044935ab31d8d4ca836665 P 7af24a9b7692461997f6e42c2b666edf: Deleting consensus metadata
I20260924 13:51:31.070581 32614 catalog_manager.cc:4841] TS 7af24a9b7692461997f6e42c2b666edf (127.13.149.194:34035): tablet 75eddaf7f0044935ab31d8d4ca836665 (table test-table [id=fe7831cb87c94a86aae4d6c84df125c6]) successfully deleted
W20260924 13:51:31.073678 32611 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.13.149.193:46591 (error 108)
I20260924 13:51:31.073884 13911 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJN9WRF/build/release/bin/kudu with pid 350
I20260924 13:51:31.074877 647 ts_tablet_manager.cc:1905] T 75eddaf7f0044935ab31d8d4ca836665 P d3bb322529d249d2b6f051c60f7d3644: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 13:51:31.075466 618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:31.075266 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.193:46591 (local address 127.0.0.1:36148)
0924 13:51:31.075375 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0924 13:51:31.075426 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.149.193:46591: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":84}
W20260924 13:51:31.075904 32611 catalog_manager.cc:4584] TS 5ad7c8617d4148dbb4e04a76cb48a7d2 (127.13.149.193:46591): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 75eddaf7f0044935ab31d8d4ca836665: Network error: Client connection negotiation failed: client connection to 127.13.149.193:46591: connect: Connection refused (error 111)
I20260924 13:51:31.076826 647 ts_tablet_manager.cc:1918] T 75eddaf7f0044935ab31d8d4ca836665 P d3bb322529d249d2b6f051c60f7d3644: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 13:51:31.076890 647 log.cc:1196] T 75eddaf7f0044935ab31d8d4ca836665 P d3bb322529d249d2b6f051c60f7d3644: Deleting WAL directory at /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790257629041645-13911-0/minicluster-data/ts-2/wal/wals/75eddaf7f0044935ab31d8d4ca836665
I20260924 13:51:31.077109 647 ts_tablet_manager.cc:1939] T 75eddaf7f0044935ab31d8d4ca836665 P d3bb322529d249d2b6f051c60f7d3644: Deleting consensus metadata
I20260924 13:51:31.077569 32612 catalog_manager.cc:4841] TS d3bb322529d249d2b6f051c60f7d3644 (127.13.149.195:46729): tablet 75eddaf7f0044935ab31d8d4ca836665 (table test-table [id=fe7831cb87c94a86aae4d6c84df125c6]) successfully deleted
I20260924 13:51:31.079181 13911 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJN9WRF/build/release/bin/kudu with pid 482
I20260924 13:51:31.084167 13911 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJN9WRF/build/release/bin/kudu with pid 32596
2026-09-24T13:51:31Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (632 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790257629041645-13911-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 13:51:31 dist-test-slave-sjl2 krb5kdc[653](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 13:51:31 dist-test-slave-sjl2 krb5kdc[653](info): set up 2 sockets
Sep 24 13:51:31 dist-test-slave-sjl2 krb5kdc[653](info): commencing operation
krb5kdc: starting...
W20260924 13:51:32.049785 13911 mini_kdc.cc:121] Time spent starting KDC: real 0.947s user 0.002s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790257629041645-13911-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790257629041645-13911-0/krb5kdc/test-user.keytab.
Sep 24 13:51:32 dist-test-slave-sjl2 krb5kdc[653](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257892, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T13:51:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T13:51:32Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.149.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.149.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.149.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.254.keytab.
Entry for principal kudu/127.13.149.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.254.keytab.
I20260924 13:51:32.165849 13911 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJN9WRF/build/release/bin/kudu
/tmp/dist-test-taskJN9WRF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790257629041645-13911-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790257629041645-13911-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790257629041645-13911-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790257629041645-13911-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.149.254:34503
--webserver_interface=127.13.149.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.254.keytab
--principal=kudu/127.13.149.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.149.212:36983
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.149.254:34503
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790257629041645-13911-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790257629041645-13911-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790257629041645-13911-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 13:51:32.246127 666 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 13:51:32.246353 666 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 13:51:32.246387 666 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 13:51:32.247611 666 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 13:51:32.247659 666 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 13:51:32.247678 666 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 13:51:32.247695 666 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 13:51:32.247710 666 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 13:51:32.249265 666 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.149.212:36983
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790257629041645-13911-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790257629041645-13911-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.149.254:34503
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.254.keytab
--principal=kudu/127.13.149.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.149.254:34503
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790257629041645-13911-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.149.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.666
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790257629041645-13911-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 12:29:37 UTC on ccycloud.quasar-iqzbxn.root.comops.site
build id 4670106
I20260924 13:51:32.249518 666 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 13:51:32.249729 666 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 13:51:32.254457 674 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 13:51:32.254602 666 server_base.cc:1034] running on GCE node
W20260924 13:51:32.254511 671 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 13:51:32.254469 672 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 13:51:32.255141 666 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 13:51:32.255419 666 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 13:51:32.256574 666 hybrid_clock.cc:648] HybridClock initialized: now 1790257892256554 us; error 37 us; skew 500 ppm
Sep 24 13:51:32 dist-test-slave-sjl2 krb5kdc[653](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257892, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.149.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 13:51:32.259039 666 init.cc:377] Logged in from keytab as kudu/127.13.149.254@KRBTEST.COM (short username kudu)
I20260924 13:51:32.259500 666 webserver.cc:460] Webserver started at http://127.13.149.254:40659/ using document root <none> and password file <none>
I20260924 13:51:32.259720 666 fs_manager.cc:359] Metadata directory not provided
I20260924 13:51:32.259775 666 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 13:51:32.259898 666 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 13:51:32.260960 666 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790257629041645-13911-0/minicluster-data/master-0/data/instance:
uuid: "1f3e7c4a1e3c41d7adec98c6ec967e45"
format_stamp: "Formatted at 2026-09-24 13:51:32 on dist-test-slave-sjl2"
server_key: "8034e250b8844041f7ec09a8cf6bf18b"
server_key_iv: "51aa94c7efe088b319159a5494617723"
server_key_version: "encryptionkey@0"
I20260924 13:51:32.261304 666 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790257629041645-13911-0/minicluster-data/master-0/wal/instance:
uuid: "1f3e7c4a1e3c41d7adec98c6ec967e45"
format_stamp: "Formatted at 2026-09-24 13:51:32 on dist-test-slave-sjl2"
server_key: "8034e250b8844041f7ec09a8cf6bf18b"
server_key_iv: "51aa94c7efe088b319159a5494617723"
server_key_version: "encryptionkey@0"
I20260924 13:51:32.262552 666 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260924 13:51:32.263289 681 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 13:51:32.263432 666 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260924 13:51:32.263511 666 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790257629041645-13911-0/minicluster-data/master-0/data,/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790257629041645-13911-0/minicluster-data/master-0/wal
uuid: "1f3e7c4a1e3c41d7adec98c6ec967e45"
format_stamp: "Formatted at 2026-09-24 13:51:32 on dist-test-slave-sjl2"
server_key: "8034e250b8844041f7ec09a8cf6bf18b"
server_key_iv: "51aa94c7efe088b319159a5494617723"
server_key_version: "encryptionkey@0"
I20260924 13:51:32.263581 666 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790257629041645-13911-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790257629041645-13911-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790257629041645-13911-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 13:51:32.272799 666 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 13:51:32.274592 666 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 13:51:32.274726 666 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 13:51:32.278586 666 rpc_server.cc:307] RPC server started. Bound to: 127.13.149.254:34503
I20260924 13:51:32.278635 733 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.149.254:34503 every 8 connection(s)
I20260924 13:51:32.278959 666 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790257629041645-13911-0/minicluster-data/master-0/data/info.pb
I20260924 13:51:32.279805 734 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 13:51:32.280752 13911 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJN9WRF/build/release/bin/kudu as pid 666
I20260924 13:51:32.280855 13911 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790257629041645-13911-0/minicluster-data/master-0/wal/instance
I20260924 13:51:32.281057 13911 external_mini_cluster.cc:1428] Setting key aa1ec87a92ae6a6bddc62382e541dba1
I20260924 13:51:32.282615 734 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1f3e7c4a1e3c41d7adec98c6ec967e45: Bootstrap starting.
I20260924 13:51:32.283221 734 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1f3e7c4a1e3c41d7adec98c6ec967e45: Neither blocks nor log segments found. Creating new log.
I20260924 13:51:32.283519 734 log.cc:824] T 00000000000000000000000000000000 P 1f3e7c4a1e3c41d7adec98c6ec967e45: Log is configured to *not* fsync() on all Append() calls
I20260924 13:51:32.284210 734 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1f3e7c4a1e3c41d7adec98c6ec967e45: No bootstrap required, opened a new log
I20260924 13:51:32.286293 734 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1f3e7c4a1e3c41d7adec98c6ec967e45 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f3e7c4a1e3c41d7adec98c6ec967e45" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 34503 } }
I20260924 13:51:32.286429 734 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1f3e7c4a1e3c41d7adec98c6ec967e45 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 13:51:32.286451 734 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1f3e7c4a1e3c41d7adec98c6ec967e45 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f3e7c4a1e3c41d7adec98c6ec967e45, State: Initialized, Role: FOLLOWER
I20260924 13:51:32.286569 734 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1f3e7c4a1e3c41d7adec98c6ec967e45 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f3e7c4a1e3c41d7adec98c6ec967e45" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 34503 } }
I20260924 13:51:32.286639 734 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1f3e7c4a1e3c41d7adec98c6ec967e45 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 13:51:32.286662 734 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1f3e7c4a1e3c41d7adec98c6ec967e45 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 13:51:32.286707 734 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1f3e7c4a1e3c41d7adec98c6ec967e45 [term 0 FOLLOWER]: Advancing to term 1
I20260924 13:51:32.287223 734 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1f3e7c4a1e3c41d7adec98c6ec967e45 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f3e7c4a1e3c41d7adec98c6ec967e45" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 34503 } }
I20260924 13:51:32.287320 734 leader_election.cc:302] T 00000000000000000000000000000000 P 1f3e7c4a1e3c41d7adec98c6ec967e45 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1f3e7c4a1e3c41d7adec98c6ec967e45; no voters:
I20260924 13:51:32.287477 734 leader_election.cc:288] T 00000000000000000000000000000000 P 1f3e7c4a1e3c41d7adec98c6ec967e45 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 13:51:32.287510 739 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1f3e7c4a1e3c41d7adec98c6ec967e45 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 13:51:32.287658 739 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1f3e7c4a1e3c41d7adec98c6ec967e45 [term 1 LEADER]: Becoming Leader. State: Replica: 1f3e7c4a1e3c41d7adec98c6ec967e45, State: Running, Role: LEADER
I20260924 13:51:32.287734 734 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1f3e7c4a1e3c41d7adec98c6ec967e45 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 13:51:32.287777 739 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1f3e7c4a1e3c41d7adec98c6ec967e45 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f3e7c4a1e3c41d7adec98c6ec967e45" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 34503 } }
Sep 24 13:51:32 dist-test-slave-sjl2 krb5kdc[653](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257892, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.149.254@KRBTEST.COM
I20260924 13:51:32.288491 741 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1f3e7c4a1e3c41d7adec98c6ec967e45 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1f3e7c4a1e3c41d7adec98c6ec967e45" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f3e7c4a1e3c41d7adec98c6ec967e45" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 34503 } } }
I20260924 13:51:32.288522 740 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1f3e7c4a1e3c41d7adec98c6ec967e45 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1f3e7c4a1e3c41d7adec98c6ec967e45. Latest consensus state: current_term: 1 leader_uuid: "1f3e7c4a1e3c41d7adec98c6ec967e45" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f3e7c4a1e3c41d7adec98c6ec967e45" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 34503 } } }
I20260924 13:51:32.288705 740 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1f3e7c4a1e3c41d7adec98c6ec967e45 [sys.catalog]: This master's current role is: LEADER
I20260924 13:51:32.288945 751 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 13:51:32.288673 741 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1f3e7c4a1e3c41d7adec98c6ec967e45 [sys.catalog]: This master's current role is: LEADER
I20260924 13:51:32.289498 751 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 13:51:32.291247 756 mvcc.cc:205] Tried to move back new op lower bound from 7332896326821679104 to 7332896326811099136. Current Snapshot: MvccSnapshot[applied={T|T < 7332896326821679104}]
I20260924 13:51:32.291941 737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:32.281541 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41112 (local address 127.13.149.254:34503)
0924 13:51:32.281728 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0924 13:51:32.281737 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0924 13:51:32.281768 (+ 31us) server_negotiation.cc:407] Connection header received
0924 13:51:32.282322 (+ 554us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:51:32.282331 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:51:32.282406 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:51:32.282555 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:51:32.283153 (+ 598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:51:32.284434 (+ 1281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:51:32.285222 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:51:32.285847 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:51:32.285965 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:51:32.288472 (+ 2507us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:51:32.288477 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:51:32.288477 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 13:51:32.288491 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:51:32.290229 (+ 1738us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:51:32.290622 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:51:32.290625 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:51:32.290625 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:51:32.290704 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:51:32.290918 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:51:32.290919 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:51:32.290920 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:51:32.291248 (+ 328us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:51:32.291440 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0924 13:51:32.291550 (+ 110us) server_negotiation.cc:299] Negotiation successful
0924 13:51:32.291728 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":136,"thread_start_us":80,"threads_started":1}
I20260924 13:51:32.293334 751 catalog_manager.cc:1347] Generated new cluster ID: 9c0767ea7c4f45ed915ccb42918f2e29
I20260924 13:51:32.293378 751 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 13:51:32.323374 751 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 13:51:32.323931 751 catalog_manager.cc:1504] Loading token signing keys...
I20260924 13:51:32.327023 751 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1f3e7c4a1e3c41d7adec98c6ec967e45: Generated new TSK 0
I20260924 13:51:32.327239 751 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.13.149.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.149.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.149.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.193.keytab.
Entry for principal kudu/127.13.149.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.193.keytab.
I20260924 13:51:32.356212 13911 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJN9WRF/build/release/bin/kudu
/tmp/dist-test-taskJN9WRF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790257629041645-13911-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790257629041645-13911-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790257629041645-13911-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790257629041645-13911-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.149.193:0
--local_ip_for_outbound_sockets=127.13.149.193
--webserver_interface=127.13.149.193
--webserver_port=0
--tserver_master_addrs=127.13.149.254:34503
--keytab_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.193.keytab
--principal=kudu/127.13.149.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.149.212:36983
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790257629041645-13911-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790257629041645-13911-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790257629041645-13911-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 13:51:32.438463 760 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 13:51:32.438685 760 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 13:51:32.438711 760 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 13:51:32.439893 760 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 13:51:32.439944 760 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 13:51:32.439971 760 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.149.193
I20260924 13:51:32.441542 760 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.149.212:36983
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790257629041645-13911-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790257629041645-13911-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.193.keytab
--principal=kudu/127.13.149.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.149.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790257629041645-13911-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.149.193
--webserver_port=0
--tserver_master_addrs=127.13.149.254:34503
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.760
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.149.193
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790257629041645-13911-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 12:29:37 UTC on ccycloud.quasar-iqzbxn.root.comops.site
build id 4670106
I20260924 13:51:32.441790 760 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 13:51:32.442034 760 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 13:51:32.447280 768 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 13:51:32.447291 766 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 13:51:32.447291 765 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 13:51:32.447322 760 server_base.cc:1034] running on GCE node
I20260924 13:51:32.447628 760 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 13:51:32.447863 760 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 13:51:32.449002 760 hybrid_clock.cc:648] HybridClock initialized: now 1790257892448983 us; error 29 us; skew 500 ppm
Sep 24 13:51:32 dist-test-slave-sjl2 krb5kdc[653](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257892, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.149.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 13:51:32.454958 760 init.cc:377] Logged in from keytab as kudu/127.13.149.193@KRBTEST.COM (short username kudu)
I20260924 13:51:32.455476 760 webserver.cc:460] Webserver started at http://127.13.149.193:41327/ using document root <none> and password file <none>
I20260924 13:51:32.455698 760 fs_manager.cc:359] Metadata directory not provided
I20260924 13:51:32.455744 760 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 13:51:32.455850 760 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 13:51:32.456904 760 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790257629041645-13911-0/minicluster-data/ts-0/data/instance:
uuid: "678eee0e73334edd8cfb3ba2874e1ec3"
format_stamp: "Formatted at 2026-09-24 13:51:32 on dist-test-slave-sjl2"
server_key: "84b83f592b21e02b2f0edec0683f0d6c"
server_key_iv: "6ff666e260513a7141b63638f0925171"
server_key_version: "encryptionkey@0"
I20260924 13:51:32.457288 760 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790257629041645-13911-0/minicluster-data/ts-0/wal/instance:
uuid: "678eee0e73334edd8cfb3ba2874e1ec3"
format_stamp: "Formatted at 2026-09-24 13:51:32 on dist-test-slave-sjl2"
server_key: "84b83f592b21e02b2f0edec0683f0d6c"
server_key_iv: "6ff666e260513a7141b63638f0925171"
server_key_version: "encryptionkey@0"
I20260924 13:51:32.458546 760 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260924 13:51:32.459295 775 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 13:51:32.459460 760 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260924 13:51:32.459542 760 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790257629041645-13911-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790257629041645-13911-0/minicluster-data/ts-0/wal
uuid: "678eee0e73334edd8cfb3ba2874e1ec3"
format_stamp: "Formatted at 2026-09-24 13:51:32 on dist-test-slave-sjl2"
server_key: "84b83f592b21e02b2f0edec0683f0d6c"
server_key_iv: "6ff666e260513a7141b63638f0925171"
server_key_version: "encryptionkey@0"
I20260924 13:51:32.459602 760 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790257629041645-13911-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790257629041645-13911-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790257629041645-13911-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 13:51:32.474571 760 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 13:51:32.476697 760 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 13:51:32.476837 760 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 13:51:32.477072 760 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 13:51:32.477434 760 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 13:51:32.477469 760 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 13:51:32.477497 760 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 13:51:32.477522 760 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 13:51:32.483140 760 rpc_server.cc:307] RPC server started. Bound to: 127.13.149.193:33509
I20260924 13:51:32.483197 888 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.149.193:33509 every 8 connection(s)
I20260924 13:51:32.483489 760 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790257629041645-13911-0/minicluster-data/ts-0/data/info.pb
Sep 24 13:51:32 dist-test-slave-sjl2 krb5kdc[653](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257892, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.149.193@KRBTEST.COM for kudu/127.13.149.254@KRBTEST.COM
I20260924 13:51:32.491045 13911 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJN9WRF/build/release/bin/kudu as pid 760
I20260924 13:51:32.491158 13911 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790257629041645-13911-0/minicluster-data/ts-0/wal/instance
I20260924 13:51:32.491350 13911 external_mini_cluster.cc:1428] Setting key ae921573010bca010524f4ea42152746
I20260924 13:51:32.493062 737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:32.484194 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.149.193:55381 (local address 127.13.149.254:34503)
0924 13:51:32.484253 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0924 13:51:32.484255 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:51:32.484496 (+ 241us) server_negotiation.cc:407] Connection header received
0924 13:51:32.484993 (+ 497us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:51:32.484995 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:51:32.485009 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:51:32.485079 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:51:32.486060 (+ 981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:51:32.486607 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:51:32.487573 (+ 966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:51:32.487834 (+ 261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:51:32.487891 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:51:32.490105 (+ 2214us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:51:32.490107 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:51:32.490107 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 13:51:32.490115 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:51:32.491547 (+ 1432us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:51:32.491961 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:51:32.491962 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:51:32.491962 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:51:32.492020 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:51:32.492357 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:51:32.492358 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:51:32.492358 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:51:32.492480 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:51:32.492535 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0924 13:51:32.492872 (+ 337us) server_negotiation.cc:299] Negotiation successful
0924 13:51:32.492989 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260924 13:51:32.493307 891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:32.484160 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:34503 (local address 127.13.149.193:55381)
0924 13:51:32.484345 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0924 13:51:32.484387 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0924 13:51:32.484877 (+ 490us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:51:32.485185 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:51:32.485188 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:51:32.485761 (+ 573us) client_negotiation.cc:189] Negotiated authn=SASL
0924 13:51:32.485964 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:51:32.485970 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:51:32.486681 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:51:32.486682 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:51:32.487483 (+ 801us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:51:32.487485 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:51:32.487935 (+ 450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:51:32.487936 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:51:32.488055 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:51:32.488528 (+ 473us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 13:51:32.488538 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0924 13:51:32.490010 (+ 1472us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 13:51:32.491640 (+ 1630us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 13:51:32.491650 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 13:51:32.491652 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 13:51:32.491880 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 13:51:32.492065 (+ 185us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 13:51:32.492066 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 13:51:32.492067 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 13:51:32.492280 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 13:51:32.492541 (+ 261us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 13:51:32.492546 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 13:51:32.492728 (+ 182us) client_negotiation.cc:769] Sending connection context
0924 13:51:32.492843 (+ 115us) client_negotiation.cc:240] Negotiation successful
0924 13:51:32.493058 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":96,"threads_started":1}
I20260924 13:51:32.494009 889 heartbeater.cc:347] Connected to a master server at 127.13.149.254:34503
I20260924 13:51:32.494107 889 heartbeater.cc:464] Registering TS with master...
I20260924 13:51:32.494305 889 heartbeater.cc:511] Master 127.13.149.254:34503 requested a full tablet report, sending...
I20260924 13:51:32.494874 698 ts_manager.cc:194] Registered new tserver with Master: 678eee0e73334edd8cfb3ba2874e1ec3 (127.13.149.193:33509)
I20260924 13:51:32.495726 698 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.149.193@KRBTEST.COM'} at 127.13.149.193:55381
WARNING: no policy specified for kudu/127.13.149.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.149.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.149.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.194.keytab.
Entry for principal kudu/127.13.149.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.194.keytab.
I20260924 13:51:32.516157 13911 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJN9WRF/build/release/bin/kudu
/tmp/dist-test-taskJN9WRF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790257629041645-13911-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790257629041645-13911-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790257629041645-13911-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790257629041645-13911-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.149.194:0
--local_ip_for_outbound_sockets=127.13.149.194
--webserver_interface=127.13.149.194
--webserver_port=0
--tserver_master_addrs=127.13.149.254:34503
--keytab_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.194.keytab
--principal=kudu/127.13.149.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.149.212:36983
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790257629041645-13911-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790257629041645-13911-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790257629041645-13911-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 13:51:32.595901 894 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 13:51:32.596114 894 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 13:51:32.596148 894 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 13:51:32.597312 894 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 13:51:32.597352 894 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 13:51:32.597378 894 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.149.194
I20260924 13:51:32.598781 894 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.149.212:36983
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790257629041645-13911-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790257629041645-13911-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.194.keytab
--principal=kudu/127.13.149.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.149.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790257629041645-13911-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.149.194
--webserver_port=0
--tserver_master_addrs=127.13.149.254:34503
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.894
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.149.194
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790257629041645-13911-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 12:29:37 UTC on ccycloud.quasar-iqzbxn.root.comops.site
build id 4670106
I20260924 13:51:32.598968 894 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 13:51:32.599157 894 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 13:51:32.603981 902 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 13:51:32.604121 899 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 13:51:32.604118 894 server_base.cc:1034] running on GCE node
W20260924 13:51:32.604152 900 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 13:51:32.604683 894 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 13:51:32.604949 894 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 13:51:32.606106 894 hybrid_clock.cc:648] HybridClock initialized: now 1790257892606073 us; error 47 us; skew 500 ppm
Sep 24 13:51:32 dist-test-slave-sjl2 krb5kdc[653](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257892, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.149.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 13:51:32.608549 894 init.cc:377] Logged in from keytab as kudu/127.13.149.194@KRBTEST.COM (short username kudu)
I20260924 13:51:32.609025 894 webserver.cc:460] Webserver started at http://127.13.149.194:34473/ using document root <none> and password file <none>
I20260924 13:51:32.609223 894 fs_manager.cc:359] Metadata directory not provided
I20260924 13:51:32.609266 894 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 13:51:32.609373 894 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 13:51:32.610392 894 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790257629041645-13911-0/minicluster-data/ts-1/data/instance:
uuid: "0a433702438c41888f9706e7886dffee"
format_stamp: "Formatted at 2026-09-24 13:51:32 on dist-test-slave-sjl2"
server_key: "88eb537bee04b8a99577aae8d07b0135"
server_key_iv: "396890993345c9ac19d3930909cd8210"
server_key_version: "encryptionkey@0"
I20260924 13:51:32.610713 894 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790257629041645-13911-0/minicluster-data/ts-1/wal/instance:
uuid: "0a433702438c41888f9706e7886dffee"
format_stamp: "Formatted at 2026-09-24 13:51:32 on dist-test-slave-sjl2"
server_key: "88eb537bee04b8a99577aae8d07b0135"
server_key_iv: "396890993345c9ac19d3930909cd8210"
server_key_version: "encryptionkey@0"
I20260924 13:51:32.612005 894 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260924 13:51:32.612807 909 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 13:51:32.613000 894 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260924 13:51:32.613092 894 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790257629041645-13911-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790257629041645-13911-0/minicluster-data/ts-1/wal
uuid: "0a433702438c41888f9706e7886dffee"
format_stamp: "Formatted at 2026-09-24 13:51:32 on dist-test-slave-sjl2"
server_key: "88eb537bee04b8a99577aae8d07b0135"
server_key_iv: "396890993345c9ac19d3930909cd8210"
server_key_version: "encryptionkey@0"
I20260924 13:51:32.613160 894 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790257629041645-13911-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790257629041645-13911-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790257629041645-13911-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 13:51:32.625501 894 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 13:51:32.627554 894 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 13:51:32.627696 894 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 13:51:32.627911 894 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 13:51:32.628201 894 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 13:51:32.628233 894 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 13:51:32.628261 894 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 13:51:32.628285 894 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 13:51:32.633908 894 rpc_server.cc:307] RPC server started. Bound to: 127.13.149.194:45893
I20260924 13:51:32.633955 1022 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.149.194:45893 every 8 connection(s)
I20260924 13:51:32.634261 894 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790257629041645-13911-0/minicluster-data/ts-1/data/info.pb
Sep 24 13:51:32 dist-test-slave-sjl2 krb5kdc[653](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257892, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.149.194@KRBTEST.COM for kudu/127.13.149.254@KRBTEST.COM
I20260924 13:51:32.640847 13911 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJN9WRF/build/release/bin/kudu as pid 894
I20260924 13:51:32.640990 13911 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790257629041645-13911-0/minicluster-data/ts-1/wal/instance
I20260924 13:51:32.641197 13911 external_mini_cluster.cc:1428] Setting key a2c17951c42e9283bf5d80c2fa512b1f
I20260924 13:51:32.643385 737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:32.634925 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.149.194:43001 (local address 127.13.149.254:34503)
0924 13:51:32.634978 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0924 13:51:32.634980 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:51:32.635159 (+ 179us) server_negotiation.cc:407] Connection header received
0924 13:51:32.635710 (+ 551us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:51:32.635711 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:51:32.635722 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:51:32.635775 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:51:32.636687 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:51:32.637202 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:51:32.638139 (+ 937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:51:32.638327 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:51:32.638381 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:51:32.640621 (+ 2240us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:51:32.640622 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:51:32.640623 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 13:51:32.640631 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:51:32.642118 (+ 1487us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:51:32.642513 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:51:32.642514 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:51:32.642514 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:51:32.642573 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:51:32.642771 (+ 198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:51:32.642771 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:51:32.642772 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:51:32.642906 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:51:32.642964 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0924 13:51:32.643206 (+ 242us) server_negotiation.cc:299] Negotiation successful
0924 13:51:32.643293 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260924 13:51:32.643589 1025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:32.634893 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:34503 (local address 127.13.149.194:43001)
0924 13:51:32.635049 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0924 13:51:32.635083 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0924 13:51:32.635600 (+ 517us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:51:32.635852 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:51:32.635855 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:51:32.636408 (+ 553us) client_negotiation.cc:189] Negotiated authn=SASL
0924 13:51:32.636595 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:51:32.636600 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:51:32.637279 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:51:32.637280 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:51:32.638060 (+ 780us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:51:32.638063 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:51:32.638393 (+ 330us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:51:32.638394 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:51:32.638506 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:51:32.638953 (+ 447us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 13:51:32.638964 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0924 13:51:32.640536 (+ 1572us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 13:51:32.642226 (+ 1690us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 13:51:32.642232 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 13:51:32.642234 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 13:51:32.642449 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 13:51:32.642625 (+ 176us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 13:51:32.642626 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 13:51:32.642626 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 13:51:32.642720 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 13:51:32.642957 (+ 237us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 13:51:32.642959 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 13:51:32.643072 (+ 113us) client_negotiation.cc:769] Sending connection context
0924 13:51:32.643206 (+ 134us) client_negotiation.cc:240] Negotiation successful
0924 13:51:32.643372 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":78,"threads_started":1}
I20260924 13:51:32.644258 1023 heartbeater.cc:347] Connected to a master server at 127.13.149.254:34503
I20260924 13:51:32.644359 1023 heartbeater.cc:464] Registering TS with master...
I20260924 13:51:32.644537 1023 heartbeater.cc:511] Master 127.13.149.254:34503 requested a full tablet report, sending...
I20260924 13:51:32.644994 698 ts_manager.cc:194] Registered new tserver with Master: 0a433702438c41888f9706e7886dffee (127.13.149.194:45893)
I20260924 13:51:32.645489 698 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.149.194@KRBTEST.COM'} at 127.13.149.194:43001
WARNING: no policy specified for kudu/127.13.149.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.149.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.149.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.195.keytab.
Entry for principal kudu/127.13.149.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.195.keytab.
I20260924 13:51:32.665766 13911 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJN9WRF/build/release/bin/kudu
/tmp/dist-test-taskJN9WRF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790257629041645-13911-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790257629041645-13911-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790257629041645-13911-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790257629041645-13911-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.149.195:0
--local_ip_for_outbound_sockets=127.13.149.195
--webserver_interface=127.13.149.195
--webserver_port=0
--tserver_master_addrs=127.13.149.254:34503
--keytab_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.195.keytab
--principal=kudu/127.13.149.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.149.212:36983
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790257629041645-13911-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790257629041645-13911-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790257629041645-13911-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 13:51:32.748114 1028 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 13:51:32.748329 1028 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 13:51:32.748353 1028 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 13:51:32.749552 1028 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 13:51:32.749599 1028 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 13:51:32.749625 1028 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.149.195
I20260924 13:51:32.751077 1028 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.149.212:36983
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790257629041645-13911-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790257629041645-13911-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.195.keytab
--principal=kudu/127.13.149.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.149.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790257629041645-13911-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.149.195
--webserver_port=0
--tserver_master_addrs=127.13.149.254:34503
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1028
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.149.195
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790257629041645-13911-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 12:29:37 UTC on ccycloud.quasar-iqzbxn.root.comops.site
build id 4670106
I20260924 13:51:32.751332 1028 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 13:51:32.751545 1028 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 13:51:32.756744 1034 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 13:51:32.756755 1033 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 13:51:32.756876 1028 server_base.cc:1034] running on GCE node
W20260924 13:51:32.756820 1036 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 13:51:32.757218 1028 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 13:51:32.757453 1028 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 13:51:32.758595 1028 hybrid_clock.cc:648] HybridClock initialized: now 1790257892758575 us; error 37 us; skew 500 ppm
Sep 24 13:51:32 dist-test-slave-sjl2 krb5kdc[653](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257892, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.149.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 13:51:32.761236 1028 init.cc:377] Logged in from keytab as kudu/127.13.149.195@KRBTEST.COM (short username kudu)
I20260924 13:51:32.761722 1028 webserver.cc:460] Webserver started at http://127.13.149.195:44627/ using document root <none> and password file <none>
I20260924 13:51:32.761945 1028 fs_manager.cc:359] Metadata directory not provided
I20260924 13:51:32.761989 1028 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 13:51:32.762099 1028 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 13:51:32.763278 1028 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790257629041645-13911-0/minicluster-data/ts-2/data/instance:
uuid: "69693531f6064660b3d8ff0e95e5fe2d"
format_stamp: "Formatted at 2026-09-24 13:51:32 on dist-test-slave-sjl2"
server_key: "792059a64e362a25b1974fe0adbbd973"
server_key_iv: "4fe92be27c00d1e2858bd4fc62756770"
server_key_version: "encryptionkey@0"
I20260924 13:51:32.763602 1028 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790257629041645-13911-0/minicluster-data/ts-2/wal/instance:
uuid: "69693531f6064660b3d8ff0e95e5fe2d"
format_stamp: "Formatted at 2026-09-24 13:51:32 on dist-test-slave-sjl2"
server_key: "792059a64e362a25b1974fe0adbbd973"
server_key_iv: "4fe92be27c00d1e2858bd4fc62756770"
server_key_version: "encryptionkey@0"
I20260924 13:51:32.764955 1028 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260924 13:51:32.765874 1044 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 13:51:32.766048 1028 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260924 13:51:32.766152 1028 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790257629041645-13911-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790257629041645-13911-0/minicluster-data/ts-2/wal
uuid: "69693531f6064660b3d8ff0e95e5fe2d"
format_stamp: "Formatted at 2026-09-24 13:51:32 on dist-test-slave-sjl2"
server_key: "792059a64e362a25b1974fe0adbbd973"
server_key_iv: "4fe92be27c00d1e2858bd4fc62756770"
server_key_version: "encryptionkey@0"
I20260924 13:51:32.766269 1028 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790257629041645-13911-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790257629041645-13911-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790257629041645-13911-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 13:51:32.783350 1028 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 13:51:32.785491 1028 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 13:51:32.785602 1028 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 13:51:32.785784 1028 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 13:51:32.786063 1028 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 13:51:32.786095 1028 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 13:51:32.786118 1028 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 13:51:32.786140 1028 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 13:51:32.791734 1028 rpc_server.cc:307] RPC server started. Bound to: 127.13.149.195:38443
I20260924 13:51:32.791832 1157 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.149.195:38443 every 8 connection(s)
I20260924 13:51:32.792106 1028 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790257629041645-13911-0/minicluster-data/ts-2/data/info.pb
Sep 24 13:51:32 dist-test-slave-sjl2 krb5kdc[653](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257892, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.149.195@KRBTEST.COM for kudu/127.13.149.254@KRBTEST.COM
I20260924 13:51:32.800767 13911 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJN9WRF/build/release/bin/kudu as pid 1028
I20260924 13:51:32.800879 13911 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790257629041645-13911-0/minicluster-data/ts-2/wal/instance
I20260924 13:51:32.801070 13911 external_mini_cluster.cc:1428] Setting key 530a738c641c000f9bbd65ca8791f359
I20260924 13:51:32.801266 737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:32.792784 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.149.195:53761 (local address 127.13.149.254:34503)
0924 13:51:32.792849 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0924 13:51:32.792851 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:51:32.793123 (+ 272us) server_negotiation.cc:407] Connection header received
0924 13:51:32.794083 (+ 960us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:51:32.794084 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:51:32.794098 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:51:32.794162 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:51:32.794805 (+ 643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:51:32.795320 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:51:32.796316 (+ 996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:51:32.796525 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:51:32.796589 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:51:32.798722 (+ 2133us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:51:32.798724 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:51:32.798725 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 13:51:32.798733 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:51:32.799849 (+ 1116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:51:32.800245 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:51:32.800246 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:51:32.800247 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:51:32.800301 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:51:32.800542 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:51:32.800543 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:51:32.800544 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:51:32.800701 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:51:32.800769 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0924 13:51:32.801085 (+ 316us) server_negotiation.cc:299] Negotiation successful
0924 13:51:32.801174 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260924 13:51:32.801483 1160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:32.792755 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:34503 (local address 127.13.149.195:53761)
0924 13:51:32.792954 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0924 13:51:32.792993 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0924 13:51:32.793956 (+ 963us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:51:32.794237 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:51:32.794241 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:51:32.794499 (+ 258us) client_negotiation.cc:189] Negotiated authn=SASL
0924 13:51:32.794706 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:51:32.794711 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:51:32.795412 (+ 701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:51:32.795413 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:51:32.796219 (+ 806us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:51:32.796222 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:51:32.796603 (+ 381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:51:32.796604 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:51:32.796731 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:51:32.797232 (+ 501us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 13:51:32.797240 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0924 13:51:32.798628 (+ 1388us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 13:51:32.799936 (+ 1308us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 13:51:32.799943 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 13:51:32.799945 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 13:51:32.800174 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 13:51:32.800355 (+ 181us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 13:51:32.800356 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 13:51:32.800356 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 13:51:32.800486 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 13:51:32.800772 (+ 286us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 13:51:32.800778 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 13:51:32.800936 (+ 158us) client_negotiation.cc:769] Sending connection context
0924 13:51:32.801046 (+ 110us) client_negotiation.cc:240] Negotiation successful
0924 13:51:32.801248 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":85,"threads_started":1}
I20260924 13:51:32.801903 1158 heartbeater.cc:347] Connected to a master server at 127.13.149.254:34503
I20260924 13:51:32.801992 1158 heartbeater.cc:464] Registering TS with master...
I20260924 13:51:32.802181 1158 heartbeater.cc:511] Master 127.13.149.254:34503 requested a full tablet report, sending...
I20260924 13:51:32.802484 698 ts_manager.cc:194] Registered new tserver with Master: 69693531f6064660b3d8ff0e95e5fe2d (127.13.149.195:38443)
I20260924 13:51:32.802829 698 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.149.195@KRBTEST.COM'} at 127.13.149.195:53761
I20260924 13:51:32.813251 13911 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260924 13:51:32.820469 737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:32.814562 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41114 (local address 127.13.149.254:34503)
0924 13:51:32.814635 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0924 13:51:32.814637 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:51:32.814667 (+ 30us) server_negotiation.cc:407] Connection header received
0924 13:51:32.814729 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:51:32.814729 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:51:32.814738 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:51:32.814784 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:51:32.815361 (+ 577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:51:32.815881 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:51:32.816629 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:51:32.816824 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:51:32.816889 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:51:32.818068 (+ 1179us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:51:32.818069 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:51:32.818070 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 13:51:32.818084 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:51:32.819357 (+ 1273us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:51:32.819691 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:51:32.819691 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:51:32.819692 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:51:32.819736 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:51:32.819899 (+ 163us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:51:32.819900 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:51:32.819901 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:51:32.820079 (+ 178us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:51:32.820190 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0924 13:51:32.820317 (+ 127us) server_negotiation.cc:299] Negotiation successful
0924 13:51:32.820397 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260924 13:51:32.822043 698 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41114:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 13:51:32.823629 698 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 13:51:32.830308 1168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:32.826069 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58004 (local address 127.13.149.195:38443)
0924 13:51:32.826233 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0924 13:51:32.826238 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 13:51:32.826553 (+ 315us) server_negotiation.cc:407] Connection header received
0924 13:51:32.826602 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:51:32.826604 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:51:32.826619 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:51:32.826914 (+ 295us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 13:51:32.826988 (+ 74us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:51:32.827754 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:51:32.829167 (+ 1413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:51:32.829911 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:51:32.829993 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:51:32.830054 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0924 13:51:32.830179 (+ 125us) server_negotiation.cc:299] Negotiation successful
0924 13:51:32.830227 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":132,"thread_start_us":99,"threads_started":1}
I20260924 13:51:32.830394 1167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:32.826185 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.193:33509 (local address 127.0.0.1:51980)
0924 13:51:32.826411 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0924 13:51:32.826419 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0924 13:51:32.826452 (+ 33us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:51:32.826756 (+ 304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:51:32.826756 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:51:32.826758 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 13:51:32.826915 (+ 157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:51:32.826917 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:51:32.828256 (+ 1339us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:51:32.828257 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:51:32.829289 (+ 1032us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:51:32.829291 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:51:32.830128 (+ 837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:51:32.830130 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:51:32.830204 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:51:32.830209 (+ 5us) client_negotiation.cc:769] Sending connection context
0924 13:51:32.830236 (+ 27us) client_negotiation.cc:240] Negotiation successful
0924 13:51:32.830293 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":215,"mutex_wait_us":117,"thread_start_us":51,"threads_started":1}
I20260924 13:51:32.830298 1169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:32.825983 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.195:38443 (local address 127.0.0.1:58004)
0924 13:51:32.826269 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0924 13:51:32.826279 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 13:51:32.826339 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:51:32.826742 (+ 403us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:51:32.826745 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:51:32.826751 (+ 6us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 13:51:32.826835 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:51:32.826837 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:51:32.827850 (+ 1013us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:51:32.827851 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:51:32.829047 (+ 1196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:51:32.829050 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:51:32.829997 (+ 947us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:51:32.829998 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:51:32.830071 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:51:32.830088 (+ 17us) client_negotiation.cc:769] Sending connection context
0924 13:51:32.830151 (+ 63us) client_negotiation.cc:240] Negotiation successful
0924 13:51:32.830211 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":272,"mutex_wait_us":7,"spinlock_wait_cycles":768,"thread_start_us":76,"threads_started":1}
I20260924 13:51:32.830579 1170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:32.826399 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51980 (local address 127.13.149.193:33509)
0924 13:51:32.826553 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0924 13:51:32.826557 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 13:51:32.826566 (+ 9us) server_negotiation.cc:407] Connection header received
0924 13:51:32.826602 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:51:32.826604 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:51:32.826619 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:51:32.826792 (+ 173us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 13:51:32.827005 (+ 213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:51:32.828143 (+ 1138us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:51:32.829363 (+ 1220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:51:32.830041 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:51:32.830346 (+ 305us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:51:32.830413 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0924 13:51:32.830441 (+ 28us) server_negotiation.cc:299] Negotiation successful
0924 13:51:32.830493 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123,"thread_start_us":80,"threads_started":1}
I20260924 13:51:32.831343 1166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:32.825983 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.194:45893 (local address 127.0.0.1:43152)
0924 13:51:32.826165 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0924 13:51:32.826204 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0924 13:51:32.826305 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:51:32.826742 (+ 437us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:51:32.826745 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:51:32.826748 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 13:51:32.826841 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:51:32.826843 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:51:32.829356 (+ 2513us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:51:32.829357 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:51:32.830043 (+ 686us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:51:32.830045 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:51:32.831134 (+ 1089us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:51:32.831135 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:51:32.831198 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:51:32.831202 (+ 4us) client_negotiation.cc:769] Sending connection context
0924 13:51:32.831230 (+ 28us) client_negotiation.cc:240] Negotiation successful
0924 13:51:32.831270 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":158,"mutex_wait_us":2,"thread_start_us":78,"threads_started":1}
I20260924 13:51:32.831235 823 tablet_service.cc:1467] Processing CreateTablet for tablet 1ded431ae50747fab70d8611601824e0 (DEFAULT_TABLE table=test-table [id=57d61cdcc117486b8d659418fb3f7af4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 13:51:32.831235 1092 tablet_service.cc:1467] Processing CreateTablet for tablet 1ded431ae50747fab70d8611601824e0 (DEFAULT_TABLE table=test-table [id=57d61cdcc117486b8d659418fb3f7af4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 13:51:32.831575 823 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1ded431ae50747fab70d8611601824e0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 13:51:32.831575 1092 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1ded431ae50747fab70d8611601824e0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 13:51:32.831725 1171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:32.826411 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43152 (local address 127.13.149.194:45893)
0924 13:51:32.826570 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0924 13:51:32.826575 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 13:51:32.826583 (+ 8us) server_negotiation.cc:407] Connection header received
0924 13:51:32.826619 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:51:32.826621 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:51:32.826634 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:51:32.827009 (+ 375us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 13:51:32.827085 (+ 76us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:51:32.828154 (+ 1069us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:51:32.830113 (+ 1959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:51:32.831049 (+ 936us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:51:32.831507 (+ 458us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:51:32.831574 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0924 13:51:32.831599 (+ 25us) server_negotiation.cc:299] Negotiation successful
0924 13:51:32.831644 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":128,"thread_start_us":94,"threads_started":1}
I20260924 13:51:32.832253 957 tablet_service.cc:1467] Processing CreateTablet for tablet 1ded431ae50747fab70d8611601824e0 (DEFAULT_TABLE table=test-table [id=57d61cdcc117486b8d659418fb3f7af4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 13:51:32.832535 957 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1ded431ae50747fab70d8611601824e0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 13:51:32.834813 1172 tablet_bootstrap.cc:492] T 1ded431ae50747fab70d8611601824e0 P 678eee0e73334edd8cfb3ba2874e1ec3: Bootstrap starting.
I20260924 13:51:32.835045 1174 tablet_bootstrap.cc:492] T 1ded431ae50747fab70d8611601824e0 P 0a433702438c41888f9706e7886dffee: Bootstrap starting.
I20260924 13:51:32.835462 1172 tablet_bootstrap.cc:654] T 1ded431ae50747fab70d8611601824e0 P 678eee0e73334edd8cfb3ba2874e1ec3: Neither blocks nor log segments found. Creating new log.
I20260924 13:51:32.835494 1173 tablet_bootstrap.cc:492] T 1ded431ae50747fab70d8611601824e0 P 69693531f6064660b3d8ff0e95e5fe2d: Bootstrap starting.
I20260924 13:51:32.835618 1174 tablet_bootstrap.cc:654] T 1ded431ae50747fab70d8611601824e0 P 0a433702438c41888f9706e7886dffee: Neither blocks nor log segments found. Creating new log.
I20260924 13:51:32.836043 1174 log.cc:824] T 1ded431ae50747fab70d8611601824e0 P 0a433702438c41888f9706e7886dffee: Log is configured to *not* fsync() on all Append() calls
I20260924 13:51:32.836031 1172 log.cc:824] T 1ded431ae50747fab70d8611601824e0 P 678eee0e73334edd8cfb3ba2874e1ec3: Log is configured to *not* fsync() on all Append() calls
I20260924 13:51:32.836083 1173 tablet_bootstrap.cc:654] T 1ded431ae50747fab70d8611601824e0 P 69693531f6064660b3d8ff0e95e5fe2d: Neither blocks nor log segments found. Creating new log.
I20260924 13:51:32.836541 1173 log.cc:824] T 1ded431ae50747fab70d8611601824e0 P 69693531f6064660b3d8ff0e95e5fe2d: Log is configured to *not* fsync() on all Append() calls
I20260924 13:51:32.836730 1174 tablet_bootstrap.cc:492] T 1ded431ae50747fab70d8611601824e0 P 0a433702438c41888f9706e7886dffee: No bootstrap required, opened a new log
I20260924 13:51:32.836800 1174 ts_tablet_manager.cc:1397] T 1ded431ae50747fab70d8611601824e0 P 0a433702438c41888f9706e7886dffee: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260924 13:51:32.836973 1172 tablet_bootstrap.cc:492] T 1ded431ae50747fab70d8611601824e0 P 678eee0e73334edd8cfb3ba2874e1ec3: No bootstrap required, opened a new log
I20260924 13:51:32.837035 1172 ts_tablet_manager.cc:1397] T 1ded431ae50747fab70d8611601824e0 P 678eee0e73334edd8cfb3ba2874e1ec3: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260924 13:51:32.837227 1173 tablet_bootstrap.cc:492] T 1ded431ae50747fab70d8611601824e0 P 69693531f6064660b3d8ff0e95e5fe2d: No bootstrap required, opened a new log
I20260924 13:51:32.837302 1173 ts_tablet_manager.cc:1397] T 1ded431ae50747fab70d8611601824e0 P 69693531f6064660b3d8ff0e95e5fe2d: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260924 13:51:32.838380 1172 raft_consensus.cc:348] T 1ded431ae50747fab70d8611601824e0 P 678eee0e73334edd8cfb3ba2874e1ec3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a433702438c41888f9706e7886dffee" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 45893 } } peers { permanent_uuid: "69693531f6064660b3d8ff0e95e5fe2d" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 38443 } } peers { permanent_uuid: "678eee0e73334edd8cfb3ba2874e1ec3" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 33509 } }
I20260924 13:51:32.838529 1172 raft_consensus.cc:374] T 1ded431ae50747fab70d8611601824e0 P 678eee0e73334edd8cfb3ba2874e1ec3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 13:51:32.838565 1172 raft_consensus.cc:729] T 1ded431ae50747fab70d8611601824e0 P 678eee0e73334edd8cfb3ba2874e1ec3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 678eee0e73334edd8cfb3ba2874e1ec3, State: Initialized, Role: FOLLOWER
I20260924 13:51:32.838692 1172 consensus_queue.cc:260] T 1ded431ae50747fab70d8611601824e0 P 678eee0e73334edd8cfb3ba2874e1ec3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a433702438c41888f9706e7886dffee" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 45893 } } peers { permanent_uuid: "69693531f6064660b3d8ff0e95e5fe2d" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 38443 } } peers { permanent_uuid: "678eee0e73334edd8cfb3ba2874e1ec3" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 33509 } }
I20260924 13:51:32.838717 1174 raft_consensus.cc:348] T 1ded431ae50747fab70d8611601824e0 P 0a433702438c41888f9706e7886dffee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a433702438c41888f9706e7886dffee" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 45893 } } peers { permanent_uuid: "69693531f6064660b3d8ff0e95e5fe2d" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 38443 } } peers { permanent_uuid: "678eee0e73334edd8cfb3ba2874e1ec3" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 33509 } }
I20260924 13:51:32.838882 1174 raft_consensus.cc:374] T 1ded431ae50747fab70d8611601824e0 P 0a433702438c41888f9706e7886dffee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 13:51:32.838917 1174 raft_consensus.cc:729] T 1ded431ae50747fab70d8611601824e0 P 0a433702438c41888f9706e7886dffee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a433702438c41888f9706e7886dffee, State: Initialized, Role: FOLLOWER
I20260924 13:51:32.838967 1172 ts_tablet_manager.cc:1428] T 1ded431ae50747fab70d8611601824e0 P 678eee0e73334edd8cfb3ba2874e1ec3: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260924 13:51:32.839020 1174 consensus_queue.cc:260] T 1ded431ae50747fab70d8611601824e0 P 0a433702438c41888f9706e7886dffee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a433702438c41888f9706e7886dffee" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 45893 } } peers { permanent_uuid: "69693531f6064660b3d8ff0e95e5fe2d" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 38443 } } peers { permanent_uuid: "678eee0e73334edd8cfb3ba2874e1ec3" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 33509 } }
I20260924 13:51:32.839095 889 heartbeater.cc:503] Master 127.13.149.254:34503 was elected leader, sending a full tablet report...
I20260924 13:51:32.839149 1173 raft_consensus.cc:348] T 1ded431ae50747fab70d8611601824e0 P 69693531f6064660b3d8ff0e95e5fe2d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a433702438c41888f9706e7886dffee" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 45893 } } peers { permanent_uuid: "69693531f6064660b3d8ff0e95e5fe2d" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 38443 } } peers { permanent_uuid: "678eee0e73334edd8cfb3ba2874e1ec3" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 33509 } }
I20260924 13:51:32.839282 1174 ts_tablet_manager.cc:1428] T 1ded431ae50747fab70d8611601824e0 P 0a433702438c41888f9706e7886dffee: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260924 13:51:32.839296 1173 raft_consensus.cc:374] T 1ded431ae50747fab70d8611601824e0 P 69693531f6064660b3d8ff0e95e5fe2d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 13:51:32.839326 1173 raft_consensus.cc:729] T 1ded431ae50747fab70d8611601824e0 P 69693531f6064660b3d8ff0e95e5fe2d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 69693531f6064660b3d8ff0e95e5fe2d, State: Initialized, Role: FOLLOWER
I20260924 13:51:32.839347 1023 heartbeater.cc:503] Master 127.13.149.254:34503 was elected leader, sending a full tablet report...
I20260924 13:51:32.839430 1173 consensus_queue.cc:260] T 1ded431ae50747fab70d8611601824e0 P 69693531f6064660b3d8ff0e95e5fe2d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a433702438c41888f9706e7886dffee" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 45893 } } peers { permanent_uuid: "69693531f6064660b3d8ff0e95e5fe2d" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 38443 } } peers { permanent_uuid: "678eee0e73334edd8cfb3ba2874e1ec3" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 33509 } }
I20260924 13:51:32.839679 1173 ts_tablet_manager.cc:1428] T 1ded431ae50747fab70d8611601824e0 P 69693531f6064660b3d8ff0e95e5fe2d: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260924 13:51:32.839793 1158 heartbeater.cc:503] Master 127.13.149.254:34503 was elected leader, sending a full tablet report...
W20260924 13:51:32.884847 1024 tablet.cc:2369] T 1ded431ae50747fab70d8611601824e0 P 0a433702438c41888f9706e7886dffee: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260924 13:51:32.886036 1180 raft_consensus.cc:482] T 1ded431ae50747fab70d8611601824e0 P 69693531f6064660b3d8ff0e95e5fe2d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 13:51:32.886154 1180 raft_consensus.cc:504] T 1ded431ae50747fab70d8611601824e0 P 69693531f6064660b3d8ff0e95e5fe2d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a433702438c41888f9706e7886dffee" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 45893 } } peers { permanent_uuid: "69693531f6064660b3d8ff0e95e5fe2d" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 38443 } } peers { permanent_uuid: "678eee0e73334edd8cfb3ba2874e1ec3" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 33509 } }
I20260924 13:51:32.886483 1180 leader_election.cc:288] T 1ded431ae50747fab70d8611601824e0 P 69693531f6064660b3d8ff0e95e5fe2d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0a433702438c41888f9706e7886dffee (127.13.149.194:45893), 678eee0e73334edd8cfb3ba2874e1ec3 (127.13.149.193:33509)
I20260924 13:51:32.889609 1171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:32.886717 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.149.195:45515 (local address 127.13.149.194:45893)
0924 13:51:32.886759 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0924 13:51:32.886761 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:51:32.886771 (+ 10us) server_negotiation.cc:407] Connection header received
0924 13:51:32.886799 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:51:32.886800 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:51:32.886811 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:51:32.886864 (+ 53us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 13:51:32.887033 (+ 169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:51:32.887639 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:51:32.888733 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:51:32.889251 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:51:32.889319 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:51:32.889327 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0924 13:51:32.889491 (+ 164us) server_negotiation.cc:299] Negotiation successful
0924 13:51:32.889534 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260924 13:51:32.889611 1160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:32.886648 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.194:45893 (local address 127.13.149.195:45515)
0924 13:51:32.886702 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0924 13:51:32.886713 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0924 13:51:32.886753 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:51:32.886883 (+ 130us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:51:32.886884 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:51:32.886885 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 13:51:32.886960 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:51:32.886962 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:51:32.887715 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:51:32.887716 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:51:32.888632 (+ 916us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:51:32.888635 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:51:32.889329 (+ 694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:51:32.889330 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:51:32.889416 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:51:32.889420 (+ 4us) client_negotiation.cc:769] Sending connection context
0924 13:51:32.889468 (+ 48us) client_negotiation.cc:240] Negotiation successful
0924 13:51:32.889514 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35}
I20260924 13:51:32.889680 1170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:32.886765 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.149.195:54607 (local address 127.13.149.193:33509)
0924 13:51:32.886809 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0924 13:51:32.886811 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:51:32.886921 (+ 110us) server_negotiation.cc:407] Connection header received
0924 13:51:32.886951 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:51:32.886952 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:51:32.886964 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:51:32.887025 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 13:51:32.887191 (+ 166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:51:32.887721 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:51:32.888810 (+ 1089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:51:32.889318 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:51:32.889387 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:51:32.889397 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0924 13:51:32.889574 (+ 177us) server_negotiation.cc:299] Negotiation successful
0924 13:51:32.889617 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260924 13:51:32.889719 1181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:32.886686 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.193:33509 (local address 127.13.149.195:54607)
0924 13:51:32.886850 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0924 13:51:32.886858 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0924 13:51:32.886910 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:51:32.887032 (+ 122us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:51:32.887033 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:51:32.887034 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 13:51:32.887099 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:51:32.887100 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:51:32.887796 (+ 696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:51:32.887797 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:51:32.888681 (+ 884us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:51:32.888684 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:51:32.889398 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:51:32.889399 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:51:32.889474 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:51:32.889479 (+ 5us) client_negotiation.cc:769] Sending connection context
0924 13:51:32.889558 (+ 79us) client_negotiation.cc:240] Negotiation successful
0924 13:51:32.889598 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":120,"threads_started":1}
I20260924 13:51:32.889974 977 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1ded431ae50747fab70d8611601824e0" candidate_uuid: "69693531f6064660b3d8ff0e95e5fe2d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a433702438c41888f9706e7886dffee" is_pre_election: true
I20260924 13:51:32.890029 843 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1ded431ae50747fab70d8611601824e0" candidate_uuid: "69693531f6064660b3d8ff0e95e5fe2d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "678eee0e73334edd8cfb3ba2874e1ec3" is_pre_election: true
I20260924 13:51:32.890131 977 raft_consensus.cc:2454] T 1ded431ae50747fab70d8611601824e0 P 0a433702438c41888f9706e7886dffee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 69693531f6064660b3d8ff0e95e5fe2d in term 0.
I20260924 13:51:32.890195 843 raft_consensus.cc:2454] T 1ded431ae50747fab70d8611601824e0 P 678eee0e73334edd8cfb3ba2874e1ec3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 69693531f6064660b3d8ff0e95e5fe2d in term 0.
I20260924 13:51:32.890367 1048 leader_election.cc:302] T 1ded431ae50747fab70d8611601824e0 P 69693531f6064660b3d8ff0e95e5fe2d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a433702438c41888f9706e7886dffee, 69693531f6064660b3d8ff0e95e5fe2d; no voters:
I20260924 13:51:32.890515 1180 raft_consensus.cc:2781] T 1ded431ae50747fab70d8611601824e0 P 69693531f6064660b3d8ff0e95e5fe2d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 13:51:32.890579 1180 raft_consensus.cc:482] T 1ded431ae50747fab70d8611601824e0 P 69693531f6064660b3d8ff0e95e5fe2d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 13:51:32.890609 1180 raft_consensus.cc:3037] T 1ded431ae50747fab70d8611601824e0 P 69693531f6064660b3d8ff0e95e5fe2d [term 0 FOLLOWER]: Advancing to term 1
I20260924 13:51:32.891301 1180 raft_consensus.cc:504] T 1ded431ae50747fab70d8611601824e0 P 69693531f6064660b3d8ff0e95e5fe2d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a433702438c41888f9706e7886dffee" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 45893 } } peers { permanent_uuid: "69693531f6064660b3d8ff0e95e5fe2d" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 38443 } } peers { permanent_uuid: "678eee0e73334edd8cfb3ba2874e1ec3" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 33509 } }
I20260924 13:51:32.891420 1180 leader_election.cc:288] T 1ded431ae50747fab70d8611601824e0 P 69693531f6064660b3d8ff0e95e5fe2d [CANDIDATE]: Term 1 election: Requested vote from peers 0a433702438c41888f9706e7886dffee (127.13.149.194:45893), 678eee0e73334edd8cfb3ba2874e1ec3 (127.13.149.193:33509)
I20260924 13:51:32.891604 843 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1ded431ae50747fab70d8611601824e0" candidate_uuid: "69693531f6064660b3d8ff0e95e5fe2d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "678eee0e73334edd8cfb3ba2874e1ec3"
I20260924 13:51:32.891671 977 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1ded431ae50747fab70d8611601824e0" candidate_uuid: "69693531f6064660b3d8ff0e95e5fe2d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a433702438c41888f9706e7886dffee"
I20260924 13:51:32.891685 843 raft_consensus.cc:3037] T 1ded431ae50747fab70d8611601824e0 P 678eee0e73334edd8cfb3ba2874e1ec3 [term 0 FOLLOWER]: Advancing to term 1
I20260924 13:51:32.891750 977 raft_consensus.cc:3037] T 1ded431ae50747fab70d8611601824e0 P 0a433702438c41888f9706e7886dffee [term 0 FOLLOWER]: Advancing to term 1
I20260924 13:51:32.892594 977 raft_consensus.cc:2454] T 1ded431ae50747fab70d8611601824e0 P 0a433702438c41888f9706e7886dffee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 69693531f6064660b3d8ff0e95e5fe2d in term 1.
I20260924 13:51:32.892594 843 raft_consensus.cc:2454] T 1ded431ae50747fab70d8611601824e0 P 678eee0e73334edd8cfb3ba2874e1ec3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 69693531f6064660b3d8ff0e95e5fe2d in term 1.
I20260924 13:51:32.892776 1047 leader_election.cc:302] T 1ded431ae50747fab70d8611601824e0 P 69693531f6064660b3d8ff0e95e5fe2d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 678eee0e73334edd8cfb3ba2874e1ec3, 69693531f6064660b3d8ff0e95e5fe2d; no voters:
I20260924 13:51:32.892874 1180 raft_consensus.cc:2781] T 1ded431ae50747fab70d8611601824e0 P 69693531f6064660b3d8ff0e95e5fe2d [term 1 FOLLOWER]: Leader election won for term 1
I20260924 13:51:32.893070 1180 raft_consensus.cc:686] T 1ded431ae50747fab70d8611601824e0 P 69693531f6064660b3d8ff0e95e5fe2d [term 1 LEADER]: Becoming Leader. State: Replica: 69693531f6064660b3d8ff0e95e5fe2d, State: Running, Role: LEADER
I20260924 13:51:32.893164 1180 consensus_queue.cc:237] T 1ded431ae50747fab70d8611601824e0 P 69693531f6064660b3d8ff0e95e5fe2d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a433702438c41888f9706e7886dffee" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 45893 } } peers { permanent_uuid: "69693531f6064660b3d8ff0e95e5fe2d" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 38443 } } peers { permanent_uuid: "678eee0e73334edd8cfb3ba2874e1ec3" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 33509 } }
I20260924 13:51:32.894025 697 catalog_manager.cc:5495] T 1ded431ae50747fab70d8611601824e0 P 69693531f6064660b3d8ff0e95e5fe2d reported cstate change: term changed from 0 to 1, leader changed from <none> to 69693531f6064660b3d8ff0e95e5fe2d (127.13.149.195). New cstate: current_term: 1 leader_uuid: "69693531f6064660b3d8ff0e95e5fe2d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a433702438c41888f9706e7886dffee" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 45893 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "69693531f6064660b3d8ff0e95e5fe2d" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 38443 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "678eee0e73334edd8cfb3ba2874e1ec3" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 33509 } health_report { overall_health: UNKNOWN } } }
I20260924 13:51:32.904706 1168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:32.902392 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58012 (local address 127.13.149.195:38443)
0924 13:51:32.902438 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0924 13:51:32.902440 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:51:32.902450 (+ 10us) server_negotiation.cc:407] Connection header received
0924 13:51:32.902532 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:51:32.902533 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:51:32.902549 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:51:32.902637 (+ 88us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 13:51:32.902758 (+ 121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:51:32.903249 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:51:32.903979 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:51:32.904181 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:51:32.904240 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:51:32.904378 (+ 138us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 13:51:32.904479 (+ 101us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 13:51:32.904547 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0924 13:51:32.904582 (+ 35us) server_negotiation.cc:299] Negotiation successful
0924 13:51:32.904608 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260924 13:51:32.906594 843 raft_consensus.cc:1261] T 1ded431ae50747fab70d8611601824e0 P 678eee0e73334edd8cfb3ba2874e1ec3 [term 1 FOLLOWER]: Refusing update from remote peer 69693531f6064660b3d8ff0e95e5fe2d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 13:51:32.906586 977 raft_consensus.cc:1261] T 1ded431ae50747fab70d8611601824e0 P 0a433702438c41888f9706e7886dffee [term 1 FOLLOWER]: Refusing update from remote peer 69693531f6064660b3d8ff0e95e5fe2d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 13:51:32.906955 1182 consensus_queue.cc:1035] T 1ded431ae50747fab70d8611601824e0 P 69693531f6064660b3d8ff0e95e5fe2d [LEADER]: Connected to new peer: Peer: permanent_uuid: "678eee0e73334edd8cfb3ba2874e1ec3" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 33509 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 13:51:32.907094 1180 consensus_queue.cc:1035] T 1ded431ae50747fab70d8611601824e0 P 69693531f6064660b3d8ff0e95e5fe2d [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a433702438c41888f9706e7886dffee" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 45893 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 13:51:32.910027 1187 mvcc.cc:205] Tried to move back new op lower bound from 7332896329342779392 to 7332896329291059200. Current Snapshot: MvccSnapshot[applied={T|T < 7332896329342779392}]
I20260924 13:51:32.911206 1186 mvcc.cc:205] Tried to move back new op lower bound from 7332896329342779392 to 7332896329291059200. Current Snapshot: MvccSnapshot[applied={T|T < 7332896329342779392 or (T in {7332896329342779392})}]
I20260924 13:51:32.911494 698 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41114:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 13:51:32.911589 698 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41114:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 13:51:32.912294 698 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1f3e7c4a1e3c41d7adec98c6ec967e45: Sending DeleteTablet for 3 replicas of tablet 1ded431ae50747fab70d8611601824e0
I20260924 13:51:32.912636 957 tablet_service.cc:1514] Processing DeleteTablet for tablet 1ded431ae50747fab70d8611601824e0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 13:51:32 UTC) from {username='kudu', principal='kudu/127.13.149.254@KRBTEST.COM'} at 127.0.0.1:43152
I20260924 13:51:32.912680 1092 tablet_service.cc:1514] Processing DeleteTablet for tablet 1ded431ae50747fab70d8611601824e0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 13:51:32 UTC) from {username='kudu', principal='kudu/127.13.149.254@KRBTEST.COM'} at 127.0.0.1:58004
I20260924 13:51:32.912760 823 tablet_service.cc:1514] Processing DeleteTablet for tablet 1ded431ae50747fab70d8611601824e0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 13:51:32 UTC) from {username='kudu', principal='kudu/127.13.149.254@KRBTEST.COM'} at 127.0.0.1:51980
I20260924 13:51:32.912955 1197 tablet_replica.cc:330] T 1ded431ae50747fab70d8611601824e0 P 678eee0e73334edd8cfb3ba2874e1ec3: stopping tablet replica
I20260924 13:51:32.913048 1197 raft_consensus.cc:2229] T 1ded431ae50747fab70d8611601824e0 P 678eee0e73334edd8cfb3ba2874e1ec3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 13:51:32.913100 13911 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJN9WRF/build/release/bin/kudu with pid 760
I20260924 13:51:32.913774 1195 tablet_replica.cc:330] T 1ded431ae50747fab70d8611601824e0 P 0a433702438c41888f9706e7886dffee: stopping tablet replica
I20260924 13:51:32.913866 1195 raft_consensus.cc:2229] T 1ded431ae50747fab70d8611601824e0 P 0a433702438c41888f9706e7886dffee [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 13:51:32.913971 1195 raft_consensus.cc:2258] T 1ded431ae50747fab70d8611601824e0 P 0a433702438c41888f9706e7886dffee [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 13:51:32.914124 1196 tablet_replica.cc:330] T 1ded431ae50747fab70d8611601824e0 P 69693531f6064660b3d8ff0e95e5fe2d: stopping tablet replica
I20260924 13:51:32.914220 1196 raft_consensus.cc:2229] T 1ded431ae50747fab70d8611601824e0 P 69693531f6064660b3d8ff0e95e5fe2d [term 1 LEADER]: Raft consensus shutting down.
I20260924 13:51:32.914306 1196 raft_consensus.cc:2258] T 1ded431ae50747fab70d8611601824e0 P 69693531f6064660b3d8ff0e95e5fe2d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 13:51:32.914664 1196 ts_tablet_manager.cc:1905] T 1ded431ae50747fab70d8611601824e0 P 69693531f6064660b3d8ff0e95e5fe2d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 13:51:32.916319 1195 ts_tablet_manager.cc:1905] T 1ded431ae50747fab70d8611601824e0 P 0a433702438c41888f9706e7886dffee: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 13:51:32.917623 1196 ts_tablet_manager.cc:1918] T 1ded431ae50747fab70d8611601824e0 P 69693531f6064660b3d8ff0e95e5fe2d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 13:51:32.917689 1196 log.cc:1196] T 1ded431ae50747fab70d8611601824e0 P 69693531f6064660b3d8ff0e95e5fe2d: Deleting WAL directory at /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790257629041645-13911-0/minicluster-data/ts-2/wal/wals/1ded431ae50747fab70d8611601824e0
I20260924 13:51:32.917900 1195 ts_tablet_manager.cc:1918] T 1ded431ae50747fab70d8611601824e0 P 0a433702438c41888f9706e7886dffee: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 13:51:32.917959 1196 ts_tablet_manager.cc:1939] T 1ded431ae50747fab70d8611601824e0 P 69693531f6064660b3d8ff0e95e5fe2d: Deleting consensus metadata
I20260924 13:51:32.918051 1195 log.cc:1196] T 1ded431ae50747fab70d8611601824e0 P 0a433702438c41888f9706e7886dffee: Deleting WAL directory at /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790257629041645-13911-0/minicluster-data/ts-1/wal/wals/1ded431ae50747fab70d8611601824e0
I20260924 13:51:32.918349 1195 ts_tablet_manager.cc:1939] T 1ded431ae50747fab70d8611601824e0 P 0a433702438c41888f9706e7886dffee: Deleting consensus metadata
I20260924 13:51:32.918350 684 catalog_manager.cc:4841] TS 69693531f6064660b3d8ff0e95e5fe2d (127.13.149.195:38443): tablet 1ded431ae50747fab70d8611601824e0 (table test-table [id=57d61cdcc117486b8d659418fb3f7af4]) successfully deleted
I20260924 13:51:32.918861 685 catalog_manager.cc:4841] TS 0a433702438c41888f9706e7886dffee (127.13.149.194:45893): tablet 1ded431ae50747fab70d8611601824e0 (table test-table [id=57d61cdcc117486b8d659418fb3f7af4]) successfully deleted
W20260924 13:51:32.919795 684 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.13.149.193:33509 (error 108)
I20260924 13:51:32.920002 13911 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJN9WRF/build/release/bin/kudu with pid 894
I20260924 13:51:32.920554 1169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:32.920130 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.193:33509 (local address 127.0.0.1:51988)
0924 13:51:32.920478 (+ 348us) negotiation.cc:107] Waiting for socket to connect
0924 13:51:32.920517 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.149.193:33509: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":328}
W20260924 13:51:32.920740 684 catalog_manager.cc:4584] TS 678eee0e73334edd8cfb3ba2874e1ec3 (127.13.149.193:33509): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 1ded431ae50747fab70d8611601824e0: Network error: Client connection negotiation failed: client connection to 127.13.149.193:33509: connect: Connection refused (error 111)
I20260924 13:51:32.926810 13911 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJN9WRF/build/release/bin/kudu with pid 1028
I20260924 13:51:32.931108 13911 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJN9WRF/build/release/bin/kudu with pid 666
2026-09-24T13:51:32Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (1846 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-09-24T13:51:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T13:51:32Z Disabled control of system clock
I20260924 13:51:32.956120 13911 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJN9WRF/build/release/bin/kudu
/tmp/dist-test-taskJN9WRF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790257629041645-13911-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790257629041645-13911-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790257629041645-13911-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790257629041645-13911-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.149.254:43311
--webserver_interface=127.13.149.254
--webserver_port=0
--builtin_ntp_servers=127.13.149.212:32859
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.149.254:43311
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260924 13:51:33.036027 1202 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 13:51:33.036278 1202 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 13:51:33.036296 1202 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 13:51:33.037464 1202 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 13:51:33.037501 1202 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 13:51:33.037513 1202 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 13:51:33.037523 1202 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 13:51:33.037533 1202 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 13:51:33.038822 1202 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.149.212:32859
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790257629041645-13911-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790257629041645-13911-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.149.254:43311
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.149.254:43311
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790257629041645-13911-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.149.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1202
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790257629041645-13911-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 12:29:37 UTC on ccycloud.quasar-iqzbxn.root.comops.site
build id 4670106
I20260924 13:51:33.039019 1202 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 13:51:33.039268 1202 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 13:51:33.043931 1207 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 13:51:33.044065 1202 server_base.cc:1034] running on GCE node
W20260924 13:51:33.043939 1210 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 13:51:33.043933 1208 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 13:51:33.044407 1202 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 13:51:33.044679 1202 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 13:51:33.045825 1202 hybrid_clock.cc:648] HybridClock initialized: now 1790257893045811 us; error 31 us; skew 500 ppm
I20260924 13:51:33.046979 1202 webserver.cc:460] Webserver started at http://127.13.149.254:43605/ using document root <none> and password file <none>
I20260924 13:51:33.047228 1202 fs_manager.cc:359] Metadata directory not provided
I20260924 13:51:33.047279 1202 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 13:51:33.047401 1202 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 13:51:33.048442 1202 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790257629041645-13911-0/minicluster-data/master-0/data/instance:
uuid: "a8787fdb70b04d75b6ad264f5f10dd35"
format_stamp: "Formatted at 2026-09-24 13:51:33 on dist-test-slave-sjl2"
server_key: "7dade9debc66e205d468825cfe616dad"
server_key_iv: "8fd6e3041bebee5404e1ed89c0460fef"
server_key_version: "encryptionkey@0"
I20260924 13:51:33.048780 1202 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790257629041645-13911-0/minicluster-data/master-0/wal/instance:
uuid: "a8787fdb70b04d75b6ad264f5f10dd35"
format_stamp: "Formatted at 2026-09-24 13:51:33 on dist-test-slave-sjl2"
server_key: "7dade9debc66e205d468825cfe616dad"
server_key_iv: "8fd6e3041bebee5404e1ed89c0460fef"
server_key_version: "encryptionkey@0"
I20260924 13:51:33.050072 1202 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260924 13:51:33.050967 1216 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 13:51:33.051137 1202 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 13:51:33.051234 1202 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790257629041645-13911-0/minicluster-data/master-0/data,/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790257629041645-13911-0/minicluster-data/master-0/wal
uuid: "a8787fdb70b04d75b6ad264f5f10dd35"
format_stamp: "Formatted at 2026-09-24 13:51:33 on dist-test-slave-sjl2"
server_key: "7dade9debc66e205d468825cfe616dad"
server_key_iv: "8fd6e3041bebee5404e1ed89c0460fef"
server_key_version: "encryptionkey@0"
I20260924 13:51:33.051312 1202 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790257629041645-13911-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790257629041645-13911-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790257629041645-13911-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 13:51:33.061260 1202 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 13:51:33.061529 1202 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 13:51:33.061626 1202 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 13:51:33.065117 1202 rpc_server.cc:307] RPC server started. Bound to: 127.13.149.254:43311
I20260924 13:51:33.065150 1268 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.149.254:43311 every 8 connection(s)
I20260924 13:51:33.065464 1202 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790257629041645-13911-0/minicluster-data/master-0/data/info.pb
I20260924 13:51:33.066377 1269 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 13:51:33.068665 1269 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a8787fdb70b04d75b6ad264f5f10dd35: Bootstrap starting.
I20260924 13:51:33.069243 1269 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a8787fdb70b04d75b6ad264f5f10dd35: Neither blocks nor log segments found. Creating new log.
I20260924 13:51:33.069500 1269 log.cc:824] T 00000000000000000000000000000000 P a8787fdb70b04d75b6ad264f5f10dd35: Log is configured to *not* fsync() on all Append() calls
I20260924 13:51:33.070181 1269 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a8787fdb70b04d75b6ad264f5f10dd35: No bootstrap required, opened a new log
I20260924 13:51:33.070295 13911 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJN9WRF/build/release/bin/kudu as pid 1202
I20260924 13:51:33.070401 13911 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790257629041645-13911-0/minicluster-data/master-0/wal/instance
I20260924 13:51:33.070603 13911 external_mini_cluster.cc:1428] Setting key 5787c3f4964cc82ffe42a876d44b4787
I20260924 13:51:33.072245 1269 raft_consensus.cc:348] T 00000000000000000000000000000000 P a8787fdb70b04d75b6ad264f5f10dd35 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a8787fdb70b04d75b6ad264f5f10dd35" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 43311 } }
I20260924 13:51:33.072412 1269 raft_consensus.cc:374] T 00000000000000000000000000000000 P a8787fdb70b04d75b6ad264f5f10dd35 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 13:51:33.072448 1269 raft_consensus.cc:729] T 00000000000000000000000000000000 P a8787fdb70b04d75b6ad264f5f10dd35 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a8787fdb70b04d75b6ad264f5f10dd35, State: Initialized, Role: FOLLOWER
I20260924 13:51:33.072577 1269 consensus_queue.cc:260] T 00000000000000000000000000000000 P a8787fdb70b04d75b6ad264f5f10dd35 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a8787fdb70b04d75b6ad264f5f10dd35" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 43311 } }
I20260924 13:51:33.072669 1269 raft_consensus.cc:388] T 00000000000000000000000000000000 P a8787fdb70b04d75b6ad264f5f10dd35 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 13:51:33.072708 1269 raft_consensus.cc:482] T 00000000000000000000000000000000 P a8787fdb70b04d75b6ad264f5f10dd35 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 13:51:33.072754 1269 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a8787fdb70b04d75b6ad264f5f10dd35 [term 0 FOLLOWER]: Advancing to term 1
I20260924 13:51:33.073458 1269 raft_consensus.cc:504] T 00000000000000000000000000000000 P a8787fdb70b04d75b6ad264f5f10dd35 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a8787fdb70b04d75b6ad264f5f10dd35" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 43311 } }
I20260924 13:51:33.073568 1269 leader_election.cc:302] T 00000000000000000000000000000000 P a8787fdb70b04d75b6ad264f5f10dd35 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a8787fdb70b04d75b6ad264f5f10dd35; no voters:
I20260924 13:51:33.073748 1269 leader_election.cc:288] T 00000000000000000000000000000000 P a8787fdb70b04d75b6ad264f5f10dd35 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 13:51:33.073787 1274 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a8787fdb70b04d75b6ad264f5f10dd35 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 13:51:33.073922 1274 raft_consensus.cc:686] T 00000000000000000000000000000000 P a8787fdb70b04d75b6ad264f5f10dd35 [term 1 LEADER]: Becoming Leader. State: Replica: a8787fdb70b04d75b6ad264f5f10dd35, State: Running, Role: LEADER
I20260924 13:51:33.073979 1269 sys_catalog.cc:564] T 00000000000000000000000000000000 P a8787fdb70b04d75b6ad264f5f10dd35 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 13:51:33.074010 1274 consensus_queue.cc:237] T 00000000000000000000000000000000 P a8787fdb70b04d75b6ad264f5f10dd35 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a8787fdb70b04d75b6ad264f5f10dd35" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 43311 } }
I20260924 13:51:33.074307 1274 sys_catalog.cc:455] T 00000000000000000000000000000000 P a8787fdb70b04d75b6ad264f5f10dd35 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a8787fdb70b04d75b6ad264f5f10dd35. Latest consensus state: current_term: 1 leader_uuid: "a8787fdb70b04d75b6ad264f5f10dd35" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a8787fdb70b04d75b6ad264f5f10dd35" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 43311 } } }
I20260924 13:51:33.074391 1274 sys_catalog.cc:458] T 00000000000000000000000000000000 P a8787fdb70b04d75b6ad264f5f10dd35 [sys.catalog]: This master's current role is: LEADER
I20260924 13:51:33.074317 1276 sys_catalog.cc:455] T 00000000000000000000000000000000 P a8787fdb70b04d75b6ad264f5f10dd35 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a8787fdb70b04d75b6ad264f5f10dd35" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a8787fdb70b04d75b6ad264f5f10dd35" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 43311 } } }
I20260924 13:51:33.074532 1276 sys_catalog.cc:458] T 00000000000000000000000000000000 P a8787fdb70b04d75b6ad264f5f10dd35 [sys.catalog]: This master's current role is: LEADER
I20260924 13:51:33.075157 1280 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 13:51:33.075623 1280 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 13:51:33.077333 1273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:33.071122 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53908 (local address 127.13.149.254:43311)
0924 13:51:33.071314 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0924 13:51:33.071323 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0924 13:51:33.071355 (+ 32us) server_negotiation.cc:407] Connection header received
0924 13:51:33.072209 (+ 854us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:51:33.072219 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:51:33.072298 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:51:33.072447 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:51:33.072673 (+ 226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:51:33.074430 (+ 1757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:51:33.075418 (+ 988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:51:33.075900 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:51:33.076008 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:51:33.076668 (+ 660us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:51:33.076671 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:51:33.076672 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 13:51:33.076869 (+ 197us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:51:33.076910 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 13:51:33.076945 (+ 35us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:51:33.077001 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0924 13:51:33.077072 (+ 71us) server_negotiation.cc:299] Negotiation successful
0924 13:51:33.077145 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":113,"thread_start_us":74,"threads_started":1}
I20260924 13:51:33.077366 1280 catalog_manager.cc:1347] Generated new cluster ID: 4548f659ba7d480fb4420b7f262a7c88
I20260924 13:51:33.077410 1280 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 13:51:33.091003 1280 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 13:51:33.091702 1280 catalog_manager.cc:1504] Loading token signing keys...
I20260924 13:51:33.095538 1280 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a8787fdb70b04d75b6ad264f5f10dd35: Generated new TSK 0
I20260924 13:51:33.095711 1280 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260924 13:51:33.098295 13911 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJN9WRF/build/release/bin/kudu
/tmp/dist-test-taskJN9WRF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790257629041645-13911-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790257629041645-13911-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790257629041645-13911-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790257629041645-13911-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.149.193:0
--local_ip_for_outbound_sockets=127.13.149.193
--webserver_interface=127.13.149.193
--webserver_port=0
--tserver_master_addrs=127.13.149.254:43311
--builtin_ntp_servers=127.13.149.212:32859
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260924 13:51:33.178105 1293 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 13:51:33.178316 1293 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 13:51:33.178349 1293 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 13:51:33.179523 1293 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 13:51:33.179563 1293 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 13:51:33.179597 1293 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.149.193
I20260924 13:51:33.181077 1293 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.149.212:32859
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790257629041645-13911-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790257629041645-13911-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.149.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790257629041645-13911-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.149.193
--webserver_port=0
--tserver_master_addrs=127.13.149.254:43311
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1293
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.149.193
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790257629041645-13911-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 12:29:37 UTC on ccycloud.quasar-iqzbxn.root.comops.site
build id 4670106
I20260924 13:51:33.181317 1293 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 13:51:33.181600 1293 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260924 13:51:33.186560 1293 server_base.cc:1034] running on GCE node
W20260924 13:51:33.186655 1301 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 13:51:33.186472 1299 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 13:51:33.186466 1298 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 13:51:33.186941 1293 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 13:51:33.187171 1293 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 13:51:33.188316 1293 hybrid_clock.cc:648] HybridClock initialized: now 1790257893188306 us; error 34 us; skew 500 ppm
I20260924 13:51:33.189597 1293 webserver.cc:460] Webserver started at http://127.13.149.193:46483/ using document root <none> and password file <none>
I20260924 13:51:33.189833 1293 fs_manager.cc:359] Metadata directory not provided
I20260924 13:51:33.189882 1293 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 13:51:33.190008 1293 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 13:51:33.191039 1293 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790257629041645-13911-0/minicluster-data/ts-0/data/instance:
uuid: "00bd7fdbbfce4930a7a81d797d203a9b"
format_stamp: "Formatted at 2026-09-24 13:51:33 on dist-test-slave-sjl2"
server_key: "4c9d6063b1460d163f1d0c56c0182b22"
server_key_iv: "60c323e626bd69403b7247bb1d395851"
server_key_version: "encryptionkey@0"
I20260924 13:51:33.191365 1293 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790257629041645-13911-0/minicluster-data/ts-0/wal/instance:
uuid: "00bd7fdbbfce4930a7a81d797d203a9b"
format_stamp: "Formatted at 2026-09-24 13:51:33 on dist-test-slave-sjl2"
server_key: "4c9d6063b1460d163f1d0c56c0182b22"
server_key_iv: "60c323e626bd69403b7247bb1d395851"
server_key_version: "encryptionkey@0"
I20260924 13:51:33.192586 1293 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260924 13:51:33.193331 1307 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 13:51:33.193500 1293 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260924 13:51:33.193606 1293 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790257629041645-13911-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790257629041645-13911-0/minicluster-data/ts-0/wal
uuid: "00bd7fdbbfce4930a7a81d797d203a9b"
format_stamp: "Formatted at 2026-09-24 13:51:33 on dist-test-slave-sjl2"
server_key: "4c9d6063b1460d163f1d0c56c0182b22"
server_key_iv: "60c323e626bd69403b7247bb1d395851"
server_key_version: "encryptionkey@0"
I20260924 13:51:33.193694 1293 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790257629041645-13911-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790257629041645-13911-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790257629041645-13911-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 13:51:33.213409 1293 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 13:51:33.213702 1293 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 13:51:33.213806 1293 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 13:51:33.213996 1293 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 13:51:33.214327 1293 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 13:51:33.214360 1293 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 13:51:33.214383 1293 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 13:51:33.214406 1293 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 13:51:33.219909 1293 rpc_server.cc:307] RPC server started. Bound to: 127.13.149.193:40385
I20260924 13:51:33.219986 1420 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.149.193:40385 every 8 connection(s)
I20260924 13:51:33.220279 1293 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790257629041645-13911-0/minicluster-data/ts-0/data/info.pb
I20260924 13:51:33.223112 13911 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJN9WRF/build/release/bin/kudu as pid 1293
I20260924 13:51:33.223255 13911 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790257629041645-13911-0/minicluster-data/ts-0/wal/instance
I20260924 13:51:33.223483 13911 external_mini_cluster.cc:1428] Setting key 66b74a499b6c273c1537267cea320108
I20260924 13:51:33.224434 13911 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJN9WRF/build/release/bin/kudu
/tmp/dist-test-taskJN9WRF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790257629041645-13911-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790257629041645-13911-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790257629041645-13911-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790257629041645-13911-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.149.194:0
--local_ip_for_outbound_sockets=127.13.149.194
--webserver_interface=127.13.149.194
--webserver_port=0
--tserver_master_addrs=127.13.149.254:43311
--builtin_ntp_servers=127.13.149.212:32859
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260924 13:51:33.225993 1273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:33.221036 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.149.193:55887 (local address 127.13.149.254:43311)
0924 13:51:33.221110 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0924 13:51:33.221113 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 13:51:33.221320 (+ 207us) server_negotiation.cc:407] Connection header received
0924 13:51:33.221835 (+ 515us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:51:33.221836 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:51:33.221859 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:51:33.221919 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:51:33.222737 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:51:33.223263 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:51:33.224457 (+ 1194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:51:33.224743 (+ 286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:51:33.224805 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:51:33.225713 (+ 908us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:51:33.225714 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:51:33.225714 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 13:51:33.225720 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:51:33.225739 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 13:51:33.225744 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:51:33.225799 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0924 13:51:33.225865 (+ 66us) server_negotiation.cc:299] Negotiation successful
0924 13:51:33.225910 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260924 13:51:33.226137 1423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:33.221003 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:43311 (local address 127.13.149.193:55887)
0924 13:51:33.221202 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0924 13:51:33.221238 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0924 13:51:33.221725 (+ 487us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:51:33.222023 (+ 298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:51:33.222026 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:51:33.222035 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0924 13:51:33.222641 (+ 606us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:51:33.222648 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:51:33.223353 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:51:33.223355 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:51:33.224351 (+ 996us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:51:33.224355 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:51:33.224828 (+ 473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:51:33.224830 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:51:33.225013 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:51:33.225537 (+ 524us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0924 13:51:33.225538 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0924 13:51:33.225579 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0924 13:51:33.225612 (+ 33us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 13:51:33.225806 (+ 194us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 13:51:33.225811 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 13:51:33.225813 (+ 2us) client_negotiation.cc:769] Sending connection context
0924 13:51:33.225865 (+ 52us) client_negotiation.cc:240] Negotiation successful
0924 13:51:33.225949 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":87,"threads_started":1}
I20260924 13:51:33.226657 1421 heartbeater.cc:347] Connected to a master server at 127.13.149.254:43311
I20260924 13:51:33.226747 1421 heartbeater.cc:464] Registering TS with master...
I20260924 13:51:33.226940 1421 heartbeater.cc:511] Master 127.13.149.254:43311 requested a full tablet report, sending...
I20260924 13:51:33.227495 1233 ts_manager.cc:194] Registered new tserver with Master: 00bd7fdbbfce4930a7a81d797d203a9b (127.13.149.193:40385)
I20260924 13:51:33.228384 1233 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.13.149.193:55887
W20260924 13:51:33.305428 1424 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 13:51:33.305624 1424 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 13:51:33.305644 1424 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 13:51:33.306715 1424 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 13:51:33.306753 1424 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 13:51:33.306777 1424 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.149.194
I20260924 13:51:33.308053 1424 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.149.212:32859
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790257629041645-13911-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790257629041645-13911-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.149.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790257629041645-13911-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.149.194
--webserver_port=0
--tserver_master_addrs=127.13.149.254:43311
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1424
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.149.194
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790257629041645-13911-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 12:29:37 UTC on ccycloud.quasar-iqzbxn.root.comops.site
build id 4670106
I20260924 13:51:33.308257 1424 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 13:51:33.308516 1424 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 13:51:33.313973 1429 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 13:51:33.313966 1430 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 13:51:33.313983 1424 server_base.cc:1034] running on GCE node
W20260924 13:51:33.313966 1432 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 13:51:33.314415 1424 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 13:51:33.314668 1424 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 13:51:33.315824 1424 hybrid_clock.cc:648] HybridClock initialized: now 1790257893315806 us; error 30 us; skew 500 ppm
I20260924 13:51:33.317216 1424 webserver.cc:460] Webserver started at http://127.13.149.194:43567/ using document root <none> and password file <none>
I20260924 13:51:33.317471 1424 fs_manager.cc:359] Metadata directory not provided
I20260924 13:51:33.317523 1424 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 13:51:33.317655 1424 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 13:51:33.318802 1424 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790257629041645-13911-0/minicluster-data/ts-1/data/instance:
uuid: "7274e9b1552f4b7e8ddcc0aae55c9edf"
format_stamp: "Formatted at 2026-09-24 13:51:33 on dist-test-slave-sjl2"
server_key: "d7f497ea8795a0d87bf1dfe9363f1e71"
server_key_iv: "533d33caf11334ad1d4cca1b178469ca"
server_key_version: "encryptionkey@0"
I20260924 13:51:33.319154 1424 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790257629041645-13911-0/minicluster-data/ts-1/wal/instance:
uuid: "7274e9b1552f4b7e8ddcc0aae55c9edf"
format_stamp: "Formatted at 2026-09-24 13:51:33 on dist-test-slave-sjl2"
server_key: "d7f497ea8795a0d87bf1dfe9363f1e71"
server_key_iv: "533d33caf11334ad1d4cca1b178469ca"
server_key_version: "encryptionkey@0"
I20260924 13:51:33.320415 1424 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260924 13:51:33.321164 1438 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 13:51:33.321316 1424 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 13:51:33.321403 1424 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790257629041645-13911-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790257629041645-13911-0/minicluster-data/ts-1/wal
uuid: "7274e9b1552f4b7e8ddcc0aae55c9edf"
format_stamp: "Formatted at 2026-09-24 13:51:33 on dist-test-slave-sjl2"
server_key: "d7f497ea8795a0d87bf1dfe9363f1e71"
server_key_iv: "533d33caf11334ad1d4cca1b178469ca"
server_key_version: "encryptionkey@0"
I20260924 13:51:33.321470 1424 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790257629041645-13911-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790257629041645-13911-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790257629041645-13911-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 13:51:33.335758 1424 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 13:51:33.336081 1424 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 13:51:33.336221 1424 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 13:51:33.336432 1424 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 13:51:33.336760 1424 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 13:51:33.336795 1424 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 13:51:33.336825 1424 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 13:51:33.336843 1424 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 13:51:33.342453 1424 rpc_server.cc:307] RPC server started. Bound to: 127.13.149.194:41145
I20260924 13:51:33.342511 1551 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.149.194:41145 every 8 connection(s)
I20260924 13:51:33.342834 1424 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790257629041645-13911-0/minicluster-data/ts-1/data/info.pb
I20260924 13:51:33.347960 1273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:33.343538 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.149.194:49237 (local address 127.13.149.254:43311)
0924 13:51:33.343589 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0924 13:51:33.343591 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:51:33.343819 (+ 228us) server_negotiation.cc:407] Connection header received
0924 13:51:33.344399 (+ 580us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:51:33.344401 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:51:33.344414 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:51:33.344471 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:51:33.345191 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:51:33.345692 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:51:33.346652 (+ 960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:51:33.346846 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:51:33.346900 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:51:33.347699 (+ 799us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:51:33.347701 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:51:33.347701 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 13:51:33.347706 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:51:33.347722 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 13:51:33.347726 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:51:33.347767 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0924 13:51:33.347845 (+ 78us) server_negotiation.cc:299] Negotiation successful
0924 13:51:33.347876 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260924 13:51:33.348176 1554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:33.343514 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:43311 (local address 127.13.149.194:49237)
0924 13:51:33.343703 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0924 13:51:33.343736 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0924 13:51:33.344285 (+ 549us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:51:33.344549 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:51:33.344554 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:51:33.344566 (+ 12us) client_negotiation.cc:189] Negotiated authn=SASL
0924 13:51:33.345101 (+ 535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:51:33.345106 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:51:33.345776 (+ 670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:51:33.345777 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:51:33.346566 (+ 789us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:51:33.346568 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:51:33.346913 (+ 345us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:51:33.346914 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:51:33.347051 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:51:33.347555 (+ 504us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0924 13:51:33.347556 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0924 13:51:33.347583 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0924 13:51:33.347606 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 13:51:33.347774 (+ 168us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 13:51:33.347779 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 13:51:33.347782 (+ 3us) client_negotiation.cc:769] Sending connection context
0924 13:51:33.347843 (+ 61us) client_negotiation.cc:240] Negotiation successful
0924 13:51:33.347960 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":79,"threads_started":1}
I20260924 13:51:33.348657 1552 heartbeater.cc:347] Connected to a master server at 127.13.149.254:43311
I20260924 13:51:33.348738 1552 heartbeater.cc:464] Registering TS with master...
I20260924 13:51:33.348903 1552 heartbeater.cc:511] Master 127.13.149.254:43311 requested a full tablet report, sending...
I20260924 13:51:33.349334 1233 ts_manager.cc:194] Registered new tserver with Master: 7274e9b1552f4b7e8ddcc0aae55c9edf (127.13.149.194:41145)
I20260924 13:51:33.349710 1233 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.13.149.194:49237
I20260924 13:51:33.350046 13911 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJN9WRF/build/release/bin/kudu as pid 1424
I20260924 13:51:33.350162 13911 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790257629041645-13911-0/minicluster-data/ts-1/wal/instance
I20260924 13:51:33.350376 13911 external_mini_cluster.cc:1428] Setting key fddebdc0adbf8af251dbf5c31c15345b
I20260924 13:51:33.351231 13911 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJN9WRF/build/release/bin/kudu
/tmp/dist-test-taskJN9WRF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790257629041645-13911-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790257629041645-13911-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790257629041645-13911-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790257629041645-13911-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.149.195:0
--local_ip_for_outbound_sockets=127.13.149.195
--webserver_interface=127.13.149.195
--webserver_port=0
--tserver_master_addrs=127.13.149.254:43311
--builtin_ntp_servers=127.13.149.212:32859
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260924 13:51:33.430176 1555 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 13:51:33.430380 1555 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 13:51:33.430400 1555 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 13:51:33.431495 1555 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 13:51:33.431528 1555 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 13:51:33.431552 1555 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.149.195
I20260924 13:51:33.432816 1555 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.149.212:32859
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790257629041645-13911-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790257629041645-13911-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.149.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790257629041645-13911-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.149.195
--webserver_port=0
--tserver_master_addrs=127.13.149.254:43311
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1555
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.149.195
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790257629041645-13911-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 12:29:37 UTC on ccycloud.quasar-iqzbxn.root.comops.site
build id 4670106
I20260924 13:51:33.433044 1555 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 13:51:33.433302 1555 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 13:51:33.437947 1561 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 13:51:33.437939 1563 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 13:51:33.438098 1555 server_base.cc:1034] running on GCE node
W20260924 13:51:33.437939 1560 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 13:51:33.438335 1555 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 13:51:33.438530 1555 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 13:51:33.439699 1555 hybrid_clock.cc:648] HybridClock initialized: now 1790257893439688 us; error 26 us; skew 500 ppm
I20260924 13:51:33.440953 1555 webserver.cc:460] Webserver started at http://127.13.149.195:38873/ using document root <none> and password file <none>
I20260924 13:51:33.441234 1555 fs_manager.cc:359] Metadata directory not provided
I20260924 13:51:33.441298 1555 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 13:51:33.441403 1555 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 13:51:33.442435 1555 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790257629041645-13911-0/minicluster-data/ts-2/data/instance:
uuid: "4c2dbd6f13924e59be894b69b3b6feda"
format_stamp: "Formatted at 2026-09-24 13:51:33 on dist-test-slave-sjl2"
server_key: "ccae2a4e5ca2249e112bb093141b8dc6"
server_key_iv: "f4097dcc99a81578f1b751244fccc8bb"
server_key_version: "encryptionkey@0"
I20260924 13:51:33.442745 1555 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790257629041645-13911-0/minicluster-data/ts-2/wal/instance:
uuid: "4c2dbd6f13924e59be894b69b3b6feda"
format_stamp: "Formatted at 2026-09-24 13:51:33 on dist-test-slave-sjl2"
server_key: "ccae2a4e5ca2249e112bb093141b8dc6"
server_key_iv: "f4097dcc99a81578f1b751244fccc8bb"
server_key_version: "encryptionkey@0"
I20260924 13:51:33.444002 1555 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260924 13:51:33.444734 1569 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 13:51:33.444885 1555 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260924 13:51:33.444993 1555 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790257629041645-13911-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790257629041645-13911-0/minicluster-data/ts-2/wal
uuid: "4c2dbd6f13924e59be894b69b3b6feda"
format_stamp: "Formatted at 2026-09-24 13:51:33 on dist-test-slave-sjl2"
server_key: "ccae2a4e5ca2249e112bb093141b8dc6"
server_key_iv: "f4097dcc99a81578f1b751244fccc8bb"
server_key_version: "encryptionkey@0"
I20260924 13:51:33.445052 1555 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790257629041645-13911-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790257629041645-13911-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790257629041645-13911-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 13:51:33.469416 1555 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 13:51:33.469748 1555 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 13:51:33.469877 1555 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 13:51:33.470113 1555 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 13:51:33.470430 1555 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 13:51:33.470464 1555 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 13:51:33.470494 1555 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 13:51:33.470518 1555 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 13:51:33.476198 1555 rpc_server.cc:307] RPC server started. Bound to: 127.13.149.195:40857
I20260924 13:51:33.476249 1682 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.149.195:40857 every 8 connection(s)
I20260924 13:51:33.476578 1555 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790257629041645-13911-0/minicluster-data/ts-2/data/info.pb
I20260924 13:51:33.480192 13911 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJN9WRF/build/release/bin/kudu as pid 1555
I20260924 13:51:33.480321 13911 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790257629041645-13911-0/minicluster-data/ts-2/wal/instance
I20260924 13:51:33.480520 13911 external_mini_cluster.cc:1428] Setting key e684006476880eb43b019ab93e31a7ec
I20260924 13:51:33.481887 1273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:33.477317 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.149.195:47679 (local address 127.13.149.254:43311)
0924 13:51:33.477367 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0924 13:51:33.477369 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:51:33.477630 (+ 261us) server_negotiation.cc:407] Connection header received
0924 13:51:33.478195 (+ 565us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:51:33.478197 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:51:33.478210 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:51:33.478262 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:51:33.478923 (+ 661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:51:33.479422 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:51:33.480387 (+ 965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:51:33.480580 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:51:33.480619 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:51:33.481596 (+ 977us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:51:33.481597 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:51:33.481597 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 13:51:33.481602 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:51:33.481618 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 13:51:33.481641 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:51:33.481688 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0924 13:51:33.481776 (+ 88us) server_negotiation.cc:299] Negotiation successful
0924 13:51:33.481809 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260924 13:51:33.482097 1685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:33.477331 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:43311 (local address 127.13.149.195:47679)
0924 13:51:33.477512 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0924 13:51:33.477547 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0924 13:51:33.478035 (+ 488us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:51:33.478327 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:51:33.478331 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:51:33.478339 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0924 13:51:33.478838 (+ 499us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:51:33.478842 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:51:33.479509 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:51:33.479510 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:51:33.480284 (+ 774us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:51:33.480286 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:51:33.480695 (+ 409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:51:33.480696 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:51:33.480838 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:51:33.481461 (+ 623us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0924 13:51:33.481462 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0924 13:51:33.481493 (+ 31us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0924 13:51:33.481517 (+ 24us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 13:51:33.481698 (+ 181us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 13:51:33.481703 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 13:51:33.481706 (+ 3us) client_negotiation.cc:769] Sending connection context
0924 13:51:33.481778 (+ 72us) client_negotiation.cc:240] Negotiation successful
0924 13:51:33.481889 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":81,"threads_started":1}
I20260924 13:51:33.482596 1683 heartbeater.cc:347] Connected to a master server at 127.13.149.254:43311
I20260924 13:51:33.482688 1683 heartbeater.cc:464] Registering TS with master...
I20260924 13:51:33.482892 1683 heartbeater.cc:511] Master 127.13.149.254:43311 requested a full tablet report, sending...
I20260924 13:51:33.483240 1233 ts_manager.cc:194] Registered new tserver with Master: 4c2dbd6f13924e59be894b69b3b6feda (127.13.149.195:40857)
I20260924 13:51:33.483604 1233 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.13.149.195:47679
I20260924 13:51:33.492460 13911 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260924 13:51:33.496701 1273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:33.493944 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53922 (local address 127.13.149.254:43311)
0924 13:51:33.494001 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0924 13:51:33.494003 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:51:33.494034 (+ 31us) server_negotiation.cc:407] Connection header received
0924 13:51:33.494110 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:51:33.494154 (+ 44us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:51:33.494166 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:51:33.494213 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:51:33.494376 (+ 163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:51:33.494859 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:51:33.495584 (+ 725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:51:33.495770 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:51:33.495823 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:51:33.496463 (+ 640us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:51:33.496464 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:51:33.496465 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 13:51:33.496469 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:51:33.496483 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 13:51:33.496486 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:51:33.496529 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0924 13:51:33.496584 (+ 55us) server_negotiation.cc:299] Negotiation successful
0924 13:51:33.496618 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260924 13:51:33.498490 1233 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:53922:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 13:51:33.500056 1233 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 13:51:33.506487 1696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:33.502636 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48464 (local address 127.13.149.195:40857)
0924 13:51:33.502867 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0924 13:51:33.502871 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 13:51:33.502880 (+ 9us) server_negotiation.cc:407] Connection header received
0924 13:51:33.502925 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:51:33.502927 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:51:33.502942 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:51:33.503007 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 13:51:33.503186 (+ 179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:51:33.504349 (+ 1163us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:51:33.505618 (+ 1269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:51:33.506098 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:51:33.506166 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:51:33.506259 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0924 13:51:33.506356 (+ 97us) server_negotiation.cc:299] Negotiation successful
0924 13:51:33.506394 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":182,"thread_start_us":76,"threads_started":1}
I20260924 13:51:33.506476 1694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:33.502349 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.195:40857 (local address 127.0.0.1:48464)
0924 13:51:33.502789 (+ 440us) negotiation.cc:107] Waiting for socket to connect
0924 13:51:33.502800 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0924 13:51:33.502867 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:51:33.503025 (+ 158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:51:33.503027 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:51:33.503029 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 13:51:33.503098 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:51:33.503100 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:51:33.504439 (+ 1339us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:51:33.504440 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:51:33.505553 (+ 1113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:51:33.505555 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:51:33.506182 (+ 627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:51:33.506183 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:51:33.506259 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:51:33.506272 (+ 13us) client_negotiation.cc:769] Sending connection context
0924 13:51:33.506341 (+ 69us) client_negotiation.cc:240] Negotiation successful
0924 13:51:33.506394 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":421,"mutex_wait_us":36,"thread_start_us":93,"threads_started":1}
I20260924 13:51:33.506659 1693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:33.502415 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34482 (local address 127.13.149.194:41145)
0924 13:51:33.502620 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0924 13:51:33.502625 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 13:51:33.502750 (+ 125us) server_negotiation.cc:407] Connection header received
0924 13:51:33.502942 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:51:33.502945 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:51:33.502960 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:51:33.503010 (+ 50us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 13:51:33.503258 (+ 248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:51:33.504343 (+ 1085us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:51:33.505399 (+ 1056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:51:33.506150 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:51:33.506213 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:51:33.506296 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0924 13:51:33.506513 (+ 217us) server_negotiation.cc:299] Negotiation successful
0924 13:51:33.506558 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":156,"thread_start_us":93,"threads_started":1}
I20260924 13:51:33.506850 1692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:33.502322 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.194:41145 (local address 127.0.0.1:34482)
0924 13:51:33.502637 (+ 315us) negotiation.cc:107] Waiting for socket to connect
0924 13:51:33.502683 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0924 13:51:33.502793 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:51:33.503146 (+ 353us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:51:33.503146 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:51:33.503151 (+ 5us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 13:51:33.503184 (+ 33us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:51:33.503186 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:51:33.504440 (+ 1254us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:51:33.504441 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:51:33.505327 (+ 886us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:51:33.505329 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:51:33.506332 (+ 1003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:51:33.506333 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:51:33.506411 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:51:33.506416 (+ 5us) client_negotiation.cc:769] Sending connection context
0924 13:51:33.506740 (+ 324us) client_negotiation.cc:240] Negotiation successful
0924 13:51:33.506784 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":265,"thread_start_us":54,"threads_started":1}
I20260924 13:51:33.507205 1695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:33.502614 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48640 (local address 127.13.149.193:40385)
0924 13:51:33.502759 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0924 13:51:33.502763 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 13:51:33.502877 (+ 114us) server_negotiation.cc:407] Connection header received
0924 13:51:33.503002 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:51:33.503004 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:51:33.503018 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:51:33.503082 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 13:51:33.503275 (+ 193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:51:33.504030 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:51:33.505438 (+ 1408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:51:33.506140 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:51:33.506212 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:51:33.506296 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0924 13:51:33.506513 (+ 217us) server_negotiation.cc:299] Negotiation successful
0924 13:51:33.507125 (+ 612us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":113,"thread_start_us":83,"threads_started":1}
I20260924 13:51:33.507170 1617 tablet_service.cc:1467] Processing CreateTablet for tablet 7ee81259be854444a9b28f4c24490fef (DEFAULT_TABLE table=test-table [id=99344944b9ee4d9bbf1506bac4199e5c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 13:51:33.507431 1691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:33.502515 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.193:40385 (local address 127.0.0.1:48640)
0924 13:51:33.502800 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0924 13:51:33.502824 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0924 13:51:33.502904 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:51:33.503145 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:51:33.503146 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:51:33.503148 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 13:51:33.503196 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:51:33.503198 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:51:33.504106 (+ 908us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:51:33.504107 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:51:33.505332 (+ 1225us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:51:33.505335 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:51:33.506333 (+ 998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:51:33.506334 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:51:33.506419 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:51:33.506421 (+ 2us) client_negotiation.cc:769] Sending connection context
0924 13:51:33.507322 (+ 901us) client_negotiation.cc:240] Negotiation successful
0924 13:51:33.507366 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":272,"mutex_wait_us":23,"thread_start_us":54,"threads_started":1}
I20260924 13:51:33.507506 1617 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7ee81259be854444a9b28f4c24490fef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 13:51:33.507503 1486 tablet_service.cc:1467] Processing CreateTablet for tablet 7ee81259be854444a9b28f4c24490fef (DEFAULT_TABLE table=test-table [id=99344944b9ee4d9bbf1506bac4199e5c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 13:51:33.507838 1486 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7ee81259be854444a9b28f4c24490fef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 13:51:33.508052 1355 tablet_service.cc:1467] Processing CreateTablet for tablet 7ee81259be854444a9b28f4c24490fef (DEFAULT_TABLE table=test-table [id=99344944b9ee4d9bbf1506bac4199e5c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 13:51:33.508371 1355 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7ee81259be854444a9b28f4c24490fef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 13:51:33.510701 1697 tablet_bootstrap.cc:492] T 7ee81259be854444a9b28f4c24490fef P 7274e9b1552f4b7e8ddcc0aae55c9edf: Bootstrap starting.
I20260924 13:51:33.511250 1697 tablet_bootstrap.cc:654] T 7ee81259be854444a9b28f4c24490fef P 7274e9b1552f4b7e8ddcc0aae55c9edf: Neither blocks nor log segments found. Creating new log.
I20260924 13:51:33.511466 1699 tablet_bootstrap.cc:492] T 7ee81259be854444a9b28f4c24490fef P 00bd7fdbbfce4930a7a81d797d203a9b: Bootstrap starting.
I20260924 13:51:33.511556 1697 log.cc:824] T 7ee81259be854444a9b28f4c24490fef P 7274e9b1552f4b7e8ddcc0aae55c9edf: Log is configured to *not* fsync() on all Append() calls
I20260924 13:51:33.511649 1698 tablet_bootstrap.cc:492] T 7ee81259be854444a9b28f4c24490fef P 4c2dbd6f13924e59be894b69b3b6feda: Bootstrap starting.
I20260924 13:51:33.512122 1699 tablet_bootstrap.cc:654] T 7ee81259be854444a9b28f4c24490fef P 00bd7fdbbfce4930a7a81d797d203a9b: Neither blocks nor log segments found. Creating new log.
I20260924 13:51:33.512250 1698 tablet_bootstrap.cc:654] T 7ee81259be854444a9b28f4c24490fef P 4c2dbd6f13924e59be894b69b3b6feda: Neither blocks nor log segments found. Creating new log.
I20260924 13:51:33.512419 1699 log.cc:824] T 7ee81259be854444a9b28f4c24490fef P 00bd7fdbbfce4930a7a81d797d203a9b: Log is configured to *not* fsync() on all Append() calls
I20260924 13:51:33.512437 1697 tablet_bootstrap.cc:492] T 7ee81259be854444a9b28f4c24490fef P 7274e9b1552f4b7e8ddcc0aae55c9edf: No bootstrap required, opened a new log
I20260924 13:51:33.512495 1697 ts_tablet_manager.cc:1397] T 7ee81259be854444a9b28f4c24490fef P 7274e9b1552f4b7e8ddcc0aae55c9edf: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260924 13:51:33.512552 1698 log.cc:824] T 7ee81259be854444a9b28f4c24490fef P 4c2dbd6f13924e59be894b69b3b6feda: Log is configured to *not* fsync() on all Append() calls
I20260924 13:51:33.513162 1699 tablet_bootstrap.cc:492] T 7ee81259be854444a9b28f4c24490fef P 00bd7fdbbfce4930a7a81d797d203a9b: No bootstrap required, opened a new log
I20260924 13:51:33.513234 1699 ts_tablet_manager.cc:1397] T 7ee81259be854444a9b28f4c24490fef P 00bd7fdbbfce4930a7a81d797d203a9b: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260924 13:51:33.514951 1697 raft_consensus.cc:348] T 7ee81259be854444a9b28f4c24490fef P 7274e9b1552f4b7e8ddcc0aae55c9edf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7274e9b1552f4b7e8ddcc0aae55c9edf" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 41145 } } peers { permanent_uuid: "00bd7fdbbfce4930a7a81d797d203a9b" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 40385 } } peers { permanent_uuid: "4c2dbd6f13924e59be894b69b3b6feda" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 40857 } }
I20260924 13:51:33.515125 1697 raft_consensus.cc:374] T 7ee81259be854444a9b28f4c24490fef P 7274e9b1552f4b7e8ddcc0aae55c9edf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 13:51:33.515162 1697 raft_consensus.cc:729] T 7ee81259be854444a9b28f4c24490fef P 7274e9b1552f4b7e8ddcc0aae55c9edf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7274e9b1552f4b7e8ddcc0aae55c9edf, State: Initialized, Role: FOLLOWER
I20260924 13:51:33.515306 1697 consensus_queue.cc:260] T 7ee81259be854444a9b28f4c24490fef P 7274e9b1552f4b7e8ddcc0aae55c9edf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7274e9b1552f4b7e8ddcc0aae55c9edf" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 41145 } } peers { permanent_uuid: "00bd7fdbbfce4930a7a81d797d203a9b" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 40385 } } peers { permanent_uuid: "4c2dbd6f13924e59be894b69b3b6feda" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 40857 } }
I20260924 13:51:33.515563 1697 ts_tablet_manager.cc:1428] T 7ee81259be854444a9b28f4c24490fef P 7274e9b1552f4b7e8ddcc0aae55c9edf: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260924 13:51:33.515561 1699 raft_consensus.cc:348] T 7ee81259be854444a9b28f4c24490fef P 00bd7fdbbfce4930a7a81d797d203a9b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7274e9b1552f4b7e8ddcc0aae55c9edf" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 41145 } } peers { permanent_uuid: "00bd7fdbbfce4930a7a81d797d203a9b" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 40385 } } peers { permanent_uuid: "4c2dbd6f13924e59be894b69b3b6feda" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 40857 } }
I20260924 13:51:33.515708 1699 raft_consensus.cc:374] T 7ee81259be854444a9b28f4c24490fef P 00bd7fdbbfce4930a7a81d797d203a9b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 13:51:33.515717 1552 heartbeater.cc:503] Master 127.13.149.254:43311 was elected leader, sending a full tablet report...
I20260924 13:51:33.515749 1699 raft_consensus.cc:729] T 7ee81259be854444a9b28f4c24490fef P 00bd7fdbbfce4930a7a81d797d203a9b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 00bd7fdbbfce4930a7a81d797d203a9b, State: Initialized, Role: FOLLOWER
I20260924 13:51:33.515856 1699 consensus_queue.cc:260] T 7ee81259be854444a9b28f4c24490fef P 00bd7fdbbfce4930a7a81d797d203a9b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7274e9b1552f4b7e8ddcc0aae55c9edf" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 41145 } } peers { permanent_uuid: "00bd7fdbbfce4930a7a81d797d203a9b" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 40385 } } peers { permanent_uuid: "4c2dbd6f13924e59be894b69b3b6feda" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 40857 } }
I20260924 13:51:33.516095 1421 heartbeater.cc:503] Master 127.13.149.254:43311 was elected leader, sending a full tablet report...
I20260924 13:51:33.516300 1699 ts_tablet_manager.cc:1428] T 7ee81259be854444a9b28f4c24490fef P 00bd7fdbbfce4930a7a81d797d203a9b: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260924 13:51:33.516702 1698 tablet_bootstrap.cc:492] T 7ee81259be854444a9b28f4c24490fef P 4c2dbd6f13924e59be894b69b3b6feda: No bootstrap required, opened a new log
I20260924 13:51:33.516772 1698 ts_tablet_manager.cc:1397] T 7ee81259be854444a9b28f4c24490fef P 4c2dbd6f13924e59be894b69b3b6feda: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20260924 13:51:33.518697 1698 raft_consensus.cc:348] T 7ee81259be854444a9b28f4c24490fef P 4c2dbd6f13924e59be894b69b3b6feda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7274e9b1552f4b7e8ddcc0aae55c9edf" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 41145 } } peers { permanent_uuid: "00bd7fdbbfce4930a7a81d797d203a9b" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 40385 } } peers { permanent_uuid: "4c2dbd6f13924e59be894b69b3b6feda" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 40857 } }
I20260924 13:51:33.518819 1698 raft_consensus.cc:374] T 7ee81259be854444a9b28f4c24490fef P 4c2dbd6f13924e59be894b69b3b6feda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 13:51:33.518838 1698 raft_consensus.cc:729] T 7ee81259be854444a9b28f4c24490fef P 4c2dbd6f13924e59be894b69b3b6feda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4c2dbd6f13924e59be894b69b3b6feda, State: Initialized, Role: FOLLOWER
I20260924 13:51:33.518919 1698 consensus_queue.cc:260] T 7ee81259be854444a9b28f4c24490fef P 4c2dbd6f13924e59be894b69b3b6feda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7274e9b1552f4b7e8ddcc0aae55c9edf" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 41145 } } peers { permanent_uuid: "00bd7fdbbfce4930a7a81d797d203a9b" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 40385 } } peers { permanent_uuid: "4c2dbd6f13924e59be894b69b3b6feda" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 40857 } }
I20260924 13:51:33.519145 1698 ts_tablet_manager.cc:1428] T 7ee81259be854444a9b28f4c24490fef P 4c2dbd6f13924e59be894b69b3b6feda: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260924 13:51:33.519204 1683 heartbeater.cc:503] Master 127.13.149.254:43311 was elected leader, sending a full tablet report...
I20260924 13:51:33.536609 1705 raft_consensus.cc:482] T 7ee81259be854444a9b28f4c24490fef P 4c2dbd6f13924e59be894b69b3b6feda [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 13:51:33.536768 1705 raft_consensus.cc:504] T 7ee81259be854444a9b28f4c24490fef P 4c2dbd6f13924e59be894b69b3b6feda [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7274e9b1552f4b7e8ddcc0aae55c9edf" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 41145 } } peers { permanent_uuid: "00bd7fdbbfce4930a7a81d797d203a9b" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 40385 } } peers { permanent_uuid: "4c2dbd6f13924e59be894b69b3b6feda" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 40857 } }
I20260924 13:51:33.537146 1705 leader_election.cc:288] T 7ee81259be854444a9b28f4c24490fef P 4c2dbd6f13924e59be894b69b3b6feda [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7274e9b1552f4b7e8ddcc0aae55c9edf (127.13.149.194:41145), 00bd7fdbbfce4930a7a81d797d203a9b (127.13.149.193:40385)
I20260924 13:51:33.540081 1685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:33.537349 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.194:41145 (local address 127.13.149.195:37017)
0924 13:51:33.537398 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0924 13:51:33.537410 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0924 13:51:33.537455 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:51:33.537660 (+ 205us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:51:33.537661 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:51:33.537663 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 13:51:33.537733 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:51:33.537735 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:51:33.538461 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:51:33.538461 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:51:33.539270 (+ 809us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:51:33.539273 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:51:33.539830 (+ 557us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:51:33.539831 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:51:33.539901 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:51:33.539905 (+ 4us) client_negotiation.cc:769] Sending connection context
0924 13:51:33.539957 (+ 52us) client_negotiation.cc:240] Negotiation successful
0924 13:51:33.540000 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":28}
I20260924 13:51:33.540128 1695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:33.537533 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.149.195:39505 (local address 127.13.149.193:40385)
0924 13:51:33.537613 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0924 13:51:33.537615 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:51:33.537697 (+ 82us) server_negotiation.cc:407] Connection header received
0924 13:51:33.537751 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:51:33.537752 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:51:33.537763 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:51:33.537817 (+ 54us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 13:51:33.537947 (+ 130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:51:33.538402 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:51:33.539358 (+ 956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:51:33.539805 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:51:33.539868 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:51:33.539875 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0924 13:51:33.540014 (+ 139us) server_negotiation.cc:299] Negotiation successful
0924 13:51:33.540059 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260924 13:51:33.540184 1693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:33.537463 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.149.195:37017 (local address 127.13.149.194:41145)
0924 13:51:33.537521 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0924 13:51:33.537523 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:51:33.537533 (+ 10us) server_negotiation.cc:407] Connection header received
0924 13:51:33.537565 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:51:33.537566 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:51:33.537575 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:51:33.537655 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 13:51:33.537804 (+ 149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:51:33.538372 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:51:33.539361 (+ 989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:51:33.539764 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:51:33.539824 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:51:33.539833 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0924 13:51:33.540070 (+ 237us) server_negotiation.cc:299] Negotiation successful
0924 13:51:33.540125 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260924 13:51:33.540092 1706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:33.537455 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.193:40385 (local address 127.13.149.195:39505)
0924 13:51:33.537637 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0924 13:51:33.537646 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0924 13:51:33.537701 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:51:33.537824 (+ 123us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:51:33.537825 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:51:33.537826 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 13:51:33.537888 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:51:33.537890 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:51:33.538485 (+ 595us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:51:33.538486 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:51:33.539266 (+ 780us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:51:33.539268 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:51:33.539892 (+ 624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:51:33.539892 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:51:33.539960 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:51:33.539962 (+ 2us) client_negotiation.cc:769] Sending connection context
0924 13:51:33.540004 (+ 42us) client_negotiation.cc:240] Negotiation successful
0924 13:51:33.540030 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":117,"threads_started":1}
I20260924 13:51:33.540503 1506 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ee81259be854444a9b28f4c24490fef" candidate_uuid: "4c2dbd6f13924e59be894b69b3b6feda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7274e9b1552f4b7e8ddcc0aae55c9edf" is_pre_election: true
I20260924 13:51:33.540643 1506 raft_consensus.cc:2454] T 7ee81259be854444a9b28f4c24490fef P 7274e9b1552f4b7e8ddcc0aae55c9edf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4c2dbd6f13924e59be894b69b3b6feda in term 0.
I20260924 13:51:33.540650 1375 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ee81259be854444a9b28f4c24490fef" candidate_uuid: "4c2dbd6f13924e59be894b69b3b6feda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "00bd7fdbbfce4930a7a81d797d203a9b" is_pre_election: true
I20260924 13:51:33.540809 1375 raft_consensus.cc:2454] T 7ee81259be854444a9b28f4c24490fef P 00bd7fdbbfce4930a7a81d797d203a9b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4c2dbd6f13924e59be894b69b3b6feda in term 0.
I20260924 13:51:33.540890 1570 leader_election.cc:302] T 7ee81259be854444a9b28f4c24490fef P 4c2dbd6f13924e59be894b69b3b6feda [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4c2dbd6f13924e59be894b69b3b6feda, 7274e9b1552f4b7e8ddcc0aae55c9edf; no voters:
I20260924 13:51:33.541049 1705 raft_consensus.cc:2781] T 7ee81259be854444a9b28f4c24490fef P 4c2dbd6f13924e59be894b69b3b6feda [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 13:51:33.541112 1705 raft_consensus.cc:482] T 7ee81259be854444a9b28f4c24490fef P 4c2dbd6f13924e59be894b69b3b6feda [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 13:51:33.541136 1705 raft_consensus.cc:3037] T 7ee81259be854444a9b28f4c24490fef P 4c2dbd6f13924e59be894b69b3b6feda [term 0 FOLLOWER]: Advancing to term 1
I20260924 13:51:33.541831 1705 raft_consensus.cc:504] T 7ee81259be854444a9b28f4c24490fef P 4c2dbd6f13924e59be894b69b3b6feda [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7274e9b1552f4b7e8ddcc0aae55c9edf" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 41145 } } peers { permanent_uuid: "00bd7fdbbfce4930a7a81d797d203a9b" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 40385 } } peers { permanent_uuid: "4c2dbd6f13924e59be894b69b3b6feda" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 40857 } }
I20260924 13:51:33.541953 1705 leader_election.cc:288] T 7ee81259be854444a9b28f4c24490fef P 4c2dbd6f13924e59be894b69b3b6feda [CANDIDATE]: Term 1 election: Requested vote from peers 7274e9b1552f4b7e8ddcc0aae55c9edf (127.13.149.194:41145), 00bd7fdbbfce4930a7a81d797d203a9b (127.13.149.193:40385)
I20260924 13:51:33.542214 1506 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ee81259be854444a9b28f4c24490fef" candidate_uuid: "4c2dbd6f13924e59be894b69b3b6feda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7274e9b1552f4b7e8ddcc0aae55c9edf"
I20260924 13:51:33.542220 1375 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ee81259be854444a9b28f4c24490fef" candidate_uuid: "4c2dbd6f13924e59be894b69b3b6feda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "00bd7fdbbfce4930a7a81d797d203a9b"
I20260924 13:51:33.542305 1375 raft_consensus.cc:3037] T 7ee81259be854444a9b28f4c24490fef P 00bd7fdbbfce4930a7a81d797d203a9b [term 0 FOLLOWER]: Advancing to term 1
I20260924 13:51:33.542305 1506 raft_consensus.cc:3037] T 7ee81259be854444a9b28f4c24490fef P 7274e9b1552f4b7e8ddcc0aae55c9edf [term 0 FOLLOWER]: Advancing to term 1
I20260924 13:51:33.543218 1506 raft_consensus.cc:2454] T 7ee81259be854444a9b28f4c24490fef P 7274e9b1552f4b7e8ddcc0aae55c9edf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4c2dbd6f13924e59be894b69b3b6feda in term 1.
I20260924 13:51:33.543218 1375 raft_consensus.cc:2454] T 7ee81259be854444a9b28f4c24490fef P 00bd7fdbbfce4930a7a81d797d203a9b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4c2dbd6f13924e59be894b69b3b6feda in term 1.
I20260924 13:51:33.543423 1570 leader_election.cc:302] T 7ee81259be854444a9b28f4c24490fef P 4c2dbd6f13924e59be894b69b3b6feda [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4c2dbd6f13924e59be894b69b3b6feda, 7274e9b1552f4b7e8ddcc0aae55c9edf; no voters:
I20260924 13:51:33.543504 1705 raft_consensus.cc:2781] T 7ee81259be854444a9b28f4c24490fef P 4c2dbd6f13924e59be894b69b3b6feda [term 1 FOLLOWER]: Leader election won for term 1
I20260924 13:51:33.543684 1705 raft_consensus.cc:686] T 7ee81259be854444a9b28f4c24490fef P 4c2dbd6f13924e59be894b69b3b6feda [term 1 LEADER]: Becoming Leader. State: Replica: 4c2dbd6f13924e59be894b69b3b6feda, State: Running, Role: LEADER
I20260924 13:51:33.543778 1705 consensus_queue.cc:237] T 7ee81259be854444a9b28f4c24490fef P 4c2dbd6f13924e59be894b69b3b6feda [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7274e9b1552f4b7e8ddcc0aae55c9edf" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 41145 } } peers { permanent_uuid: "00bd7fdbbfce4930a7a81d797d203a9b" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 40385 } } peers { permanent_uuid: "4c2dbd6f13924e59be894b69b3b6feda" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 40857 } }
I20260924 13:51:33.544595 1233 catalog_manager.cc:5495] T 7ee81259be854444a9b28f4c24490fef P 4c2dbd6f13924e59be894b69b3b6feda reported cstate change: term changed from 0 to 1, leader changed from <none> to 4c2dbd6f13924e59be894b69b3b6feda (127.13.149.195). New cstate: current_term: 1 leader_uuid: "4c2dbd6f13924e59be894b69b3b6feda" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7274e9b1552f4b7e8ddcc0aae55c9edf" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 41145 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "00bd7fdbbfce4930a7a81d797d203a9b" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 40385 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4c2dbd6f13924e59be894b69b3b6feda" member_type: VOTER last_known_addr { host: "127.13.149.195" port: 40857 } health_report { overall_health: HEALTHY } } }
I20260924 13:51:33.554237 1696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:33.551779 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48480 (local address 127.13.149.195:40857)
0924 13:51:33.551832 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0924 13:51:33.551833 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0924 13:51:33.551842 (+ 9us) server_negotiation.cc:407] Connection header received
0924 13:51:33.551982 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:51:33.551983 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:51:33.552001 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:51:33.552070 (+ 69us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 13:51:33.552214 (+ 144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:51:33.552678 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:51:33.553501 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:51:33.553712 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:51:33.553766 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:51:33.553866 (+ 100us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 13:51:33.553972 (+ 106us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 13:51:33.554024 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0924 13:51:33.554126 (+ 102us) server_negotiation.cc:299] Negotiation successful
0924 13:51:33.554143 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260924 13:51:33.556157 1375 raft_consensus.cc:1261] T 7ee81259be854444a9b28f4c24490fef P 00bd7fdbbfce4930a7a81d797d203a9b [term 1 FOLLOWER]: Refusing update from remote peer 4c2dbd6f13924e59be894b69b3b6feda: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 13:51:33.556196 1506 raft_consensus.cc:1261] T 7ee81259be854444a9b28f4c24490fef P 7274e9b1552f4b7e8ddcc0aae55c9edf [term 1 FOLLOWER]: Refusing update from remote peer 4c2dbd6f13924e59be894b69b3b6feda: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 13:51:33.556571 1707 consensus_queue.cc:1035] T 7ee81259be854444a9b28f4c24490fef P 4c2dbd6f13924e59be894b69b3b6feda [LEADER]: Connected to new peer: Peer: permanent_uuid: "7274e9b1552f4b7e8ddcc0aae55c9edf" member_type: VOTER last_known_addr { host: "127.13.149.194" port: 41145 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 13:51:33.556722 1705 consensus_queue.cc:1035] T 7ee81259be854444a9b28f4c24490fef P 4c2dbd6f13924e59be894b69b3b6feda [LEADER]: Connected to new peer: Peer: permanent_uuid: "00bd7fdbbfce4930a7a81d797d203a9b" member_type: VOTER last_known_addr { host: "127.13.149.193" port: 40385 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 13:51:33.559378 1711 mvcc.cc:205] Tried to move back new op lower bound from 7332896332003237888 to 7332896331956027392. Current Snapshot: MvccSnapshot[applied={T|T < 7332896332003237888}]
I20260924 13:51:33.561558 1233 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:53922:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 13:51:33.561650 1233 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:53922:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 13:51:33.562418 1233 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a8787fdb70b04d75b6ad264f5f10dd35: Sending DeleteTablet for 3 replicas of tablet 7ee81259be854444a9b28f4c24490fef
I20260924 13:51:33.562727 1617 tablet_service.cc:1514] Processing DeleteTablet for tablet 7ee81259be854444a9b28f4c24490fef with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 13:51:33 UTC) from {username='slave'} at 127.0.0.1:48464
I20260924 13:51:33.562798 1355 tablet_service.cc:1514] Processing DeleteTablet for tablet 7ee81259be854444a9b28f4c24490fef with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 13:51:33 UTC) from {username='slave'} at 127.0.0.1:48640
I20260924 13:51:33.562803 1486 tablet_service.cc:1514] Processing DeleteTablet for tablet 7ee81259be854444a9b28f4c24490fef with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 13:51:33 UTC) from {username='slave'} at 127.0.0.1:34482
I20260924 13:51:33.562964 1720 tablet_replica.cc:330] T 7ee81259be854444a9b28f4c24490fef P 4c2dbd6f13924e59be894b69b3b6feda: stopping tablet replica
I20260924 13:51:33.563052 1721 tablet_replica.cc:330] T 7ee81259be854444a9b28f4c24490fef P 00bd7fdbbfce4930a7a81d797d203a9b: stopping tablet replica
I20260924 13:51:33.563109 1720 raft_consensus.cc:2229] T 7ee81259be854444a9b28f4c24490fef P 4c2dbd6f13924e59be894b69b3b6feda [term 1 LEADER]: Raft consensus shutting down.
I20260924 13:51:33.563150 1721 raft_consensus.cc:2229] T 7ee81259be854444a9b28f4c24490fef P 00bd7fdbbfce4930a7a81d797d203a9b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 13:51:33.563230 1720 raft_consensus.cc:2258] T 7ee81259be854444a9b28f4c24490fef P 4c2dbd6f13924e59be894b69b3b6feda [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 13:51:33.563252 1721 raft_consensus.cc:2258] T 7ee81259be854444a9b28f4c24490fef P 00bd7fdbbfce4930a7a81d797d203a9b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 13:51:33.563516 1722 tablet_replica.cc:330] T 7ee81259be854444a9b28f4c24490fef P 7274e9b1552f4b7e8ddcc0aae55c9edf: stopping tablet replica
I20260924 13:51:33.563603 13911 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJN9WRF/build/release/bin/kudu with pid 1293
I20260924 13:51:33.563675 1720 ts_tablet_manager.cc:1905] T 7ee81259be854444a9b28f4c24490fef P 4c2dbd6f13924e59be894b69b3b6feda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 13:51:33.565541 1720 ts_tablet_manager.cc:1918] T 7ee81259be854444a9b28f4c24490fef P 4c2dbd6f13924e59be894b69b3b6feda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 13:51:33.565702 1720 log.cc:1196] T 7ee81259be854444a9b28f4c24490fef P 4c2dbd6f13924e59be894b69b3b6feda: Deleting WAL directory at /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790257629041645-13911-0/minicluster-data/ts-2/wal/wals/7ee81259be854444a9b28f4c24490fef
I20260924 13:51:33.566010 1720 ts_tablet_manager.cc:1939] T 7ee81259be854444a9b28f4c24490fef P 4c2dbd6f13924e59be894b69b3b6feda: Deleting consensus metadata
I20260924 13:51:33.566439 1218 catalog_manager.cc:4841] TS 4c2dbd6f13924e59be894b69b3b6feda (127.13.149.195:40857): tablet 7ee81259be854444a9b28f4c24490fef (table test-table [id=99344944b9ee4d9bbf1506bac4199e5c]) successfully deleted
W20260924 13:51:33.568881 1217 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.13.149.193:40385 (error 108)
I20260924 13:51:33.569090 1722 raft_consensus.cc:2229] T 7ee81259be854444a9b28f4c24490fef P 7274e9b1552f4b7e8ddcc0aae55c9edf [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 13:51:33.569248 1722 raft_consensus.cc:2258] T 7ee81259be854444a9b28f4c24490fef P 7274e9b1552f4b7e8ddcc0aae55c9edf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 13:51:33.569530 13911 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJN9WRF/build/release/bin/kudu with pid 1424
I20260924 13:51:33.570108 1691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:33.569536 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.193:40385 (local address 127.0.0.1:48642)
0924 13:51:33.570017 (+ 481us) negotiation.cc:107] Waiting for socket to connect
0924 13:51:33.570070 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.149.193:40385: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":455}
W20260924 13:51:33.571502 1217 catalog_manager.cc:4584] TS 00bd7fdbbfce4930a7a81d797d203a9b (127.13.149.193:40385): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 7ee81259be854444a9b28f4c24490fef: Network error: Client connection negotiation failed: client connection to 127.13.149.193:40385: connect: Connection refused (error 111)
I20260924 13:51:33.575986 13911 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJN9WRF/build/release/bin/kudu with pid 1555
I20260924 13:51:33.576108 1691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:33.575988 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.194:41145 (local address 127.0.0.1:34484)
0924 13:51:33.576038 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0924 13:51:33.576067 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.149.194:41145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
I20260924 13:51:33.581029 13911 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJN9WRF/build/release/bin/kudu with pid 1202
2026-09-24T13:51:33Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (649 ms)
[----------] 6 tests from EncryptionPolicyTest (7444 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790257629041645-13911-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 13:51:33 dist-test-slave-sjl2 krb5kdc[1726](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 13:51:33 dist-test-slave-sjl2 krb5kdc[1726](info): set up 2 sockets
Sep 24 13:51:33 dist-test-slave-sjl2 krb5kdc[1726](info): commencing operation
krb5kdc: starting...
W20260924 13:51:34.525889 13911 mini_kdc.cc:121] Time spent starting KDC: real 0.926s user 0.005s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790257629041645-13911-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790257629041645-13911-0/krb5kdc/test-user.keytab.
Sep 24 13:51:34 dist-test-slave-sjl2 krb5kdc[1726](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257894, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T13:51:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T13:51:34Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790257629041645-13911-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790257629041645-13911-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260924 13:51:34.644032 13911 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJN9WRF/build/release/bin/kudu
/tmp/dist-test-taskJN9WRF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790257629041645-13911-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790257629041645-13911-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790257629041645-13911-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790257629041645-13911-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35977
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790257629041645-13911-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33489
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35977
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790257629041645-13911-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790257629041645-13911-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790257629041645-13911-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 13:51:34.722895 1739 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 13:51:34.723122 1739 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 13:51:34.723145 1739 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 13:51:34.724324 1739 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 13:51:34.724371 1739 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 13:51:34.724385 1739 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 13:51:34.724395 1739 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 13:51:34.724407 1739 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 13:51:34.726004 1739 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33489
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790257629041645-13911-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790257629041645-13911-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35977
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790257629041645-13911-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35977
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790257629041645-13911-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1739
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790257629041645-13911-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 12:29:37 UTC on ccycloud.quasar-iqzbxn.root.comops.site
build id 4670106
I20260924 13:51:34.726254 1739 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 13:51:34.726485 1739 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 13:51:34.731256 1744 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 13:51:34.731238 1747 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 13:51:34.731333 1739 server_base.cc:1034] running on GCE node
W20260924 13:51:34.731238 1745 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 13:51:34.731707 1739 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 13:51:34.731990 1739 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 13:51:34.733130 1739 hybrid_clock.cc:648] HybridClock initialized: now 1790257894733109 us; error 34 us; skew 500 ppm
Sep 24 13:51:34 dist-test-slave-sjl2 krb5kdc[1726](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257894, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 13:51:34.735461 1739 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260924 13:51:34.735895 1739 webserver.cc:460] Webserver started at http://127.0.0.1:44247/ using document root <none> and password file <none>
I20260924 13:51:34.736083 1739 fs_manager.cc:359] Metadata directory not provided
I20260924 13:51:34.736122 1739 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 13:51:34.736213 1739 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 13:51:34.737243 1739 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790257629041645-13911-0/minicluster-data/master-0/data/instance:
uuid: "b302de404f8845cf9f084d646e1b18ae"
format_stamp: "Formatted at 2026-09-24 13:51:34 on dist-test-slave-sjl2"
server_key: "bf4637720e1d37302da861bc70508895"
server_key_iv: "744334c4313a03129b6b8020b3c3df56"
server_key_version: "encryptionkey@0"
I20260924 13:51:34.737535 1739 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790257629041645-13911-0/minicluster-data/master-0/wal/instance:
uuid: "b302de404f8845cf9f084d646e1b18ae"
format_stamp: "Formatted at 2026-09-24 13:51:34 on dist-test-slave-sjl2"
server_key: "bf4637720e1d37302da861bc70508895"
server_key_iv: "744334c4313a03129b6b8020b3c3df56"
server_key_version: "encryptionkey@0"
I20260924 13:51:34.738744 1739 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260924 13:51:34.739444 1754 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 13:51:34.739593 1739 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 13:51:34.739665 1739 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790257629041645-13911-0/minicluster-data/master-0/data,/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790257629041645-13911-0/minicluster-data/master-0/wal
uuid: "b302de404f8845cf9f084d646e1b18ae"
format_stamp: "Formatted at 2026-09-24 13:51:34 on dist-test-slave-sjl2"
server_key: "bf4637720e1d37302da861bc70508895"
server_key_iv: "744334c4313a03129b6b8020b3c3df56"
server_key_version: "encryptionkey@0"
I20260924 13:51:34.739717 1739 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790257629041645-13911-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790257629041645-13911-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790257629041645-13911-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 13:51:34.758772 1739 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 13:51:34.760771 1739 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 13:51:34.760885 1739 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 13:51:34.764712 1739 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35977
I20260924 13:51:34.764830 1806 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35977 every 8 connection(s)
I20260924 13:51:34.765115 1739 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790257629041645-13911-0/minicluster-data/master-0/data/info.pb
I20260924 13:51:34.766019 1807 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 13:51:34.768409 1807 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b302de404f8845cf9f084d646e1b18ae: Bootstrap starting.
I20260924 13:51:34.768874 13911 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJN9WRF/build/release/bin/kudu as pid 1739
I20260924 13:51:34.769019 13911 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790257629041645-13911-0/minicluster-data/master-0/wal/instance
I20260924 13:51:34.769075 1807 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b302de404f8845cf9f084d646e1b18ae: Neither blocks nor log segments found. Creating new log.
I20260924 13:51:34.769259 13911 external_mini_cluster.cc:1428] Setting key 956c1d5824371d1a07824b965a7aa2bf
I20260924 13:51:34.769367 1807 log.cc:824] T 00000000000000000000000000000000 P b302de404f8845cf9f084d646e1b18ae: Log is configured to *not* fsync() on all Append() calls
I20260924 13:51:34.770141 1807 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b302de404f8845cf9f084d646e1b18ae: No bootstrap required, opened a new log
I20260924 13:51:34.772524 1807 raft_consensus.cc:348] T 00000000000000000000000000000000 P b302de404f8845cf9f084d646e1b18ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b302de404f8845cf9f084d646e1b18ae" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35977 } }
I20260924 13:51:34.772701 1807 raft_consensus.cc:374] T 00000000000000000000000000000000 P b302de404f8845cf9f084d646e1b18ae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 13:51:34.772729 1807 raft_consensus.cc:729] T 00000000000000000000000000000000 P b302de404f8845cf9f084d646e1b18ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b302de404f8845cf9f084d646e1b18ae, State: Initialized, Role: FOLLOWER
I20260924 13:51:34.772850 1807 consensus_queue.cc:260] T 00000000000000000000000000000000 P b302de404f8845cf9f084d646e1b18ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b302de404f8845cf9f084d646e1b18ae" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35977 } }
I20260924 13:51:34.772948 1807 raft_consensus.cc:388] T 00000000000000000000000000000000 P b302de404f8845cf9f084d646e1b18ae [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 13:51:34.772985 1807 raft_consensus.cc:482] T 00000000000000000000000000000000 P b302de404f8845cf9f084d646e1b18ae [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 13:51:34.773029 1807 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b302de404f8845cf9f084d646e1b18ae [term 0 FOLLOWER]: Advancing to term 1
I20260924 13:51:34.773552 1807 raft_consensus.cc:504] T 00000000000000000000000000000000 P b302de404f8845cf9f084d646e1b18ae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b302de404f8845cf9f084d646e1b18ae" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35977 } }
I20260924 13:51:34.773658 1807 leader_election.cc:302] T 00000000000000000000000000000000 P b302de404f8845cf9f084d646e1b18ae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b302de404f8845cf9f084d646e1b18ae; no voters:
I20260924 13:51:34.773818 1807 leader_election.cc:288] T 00000000000000000000000000000000 P b302de404f8845cf9f084d646e1b18ae [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 13:51:34.773871 1812 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b302de404f8845cf9f084d646e1b18ae [term 1 FOLLOWER]: Leader election won for term 1
I20260924 13:51:34.774060 1812 raft_consensus.cc:686] T 00000000000000000000000000000000 P b302de404f8845cf9f084d646e1b18ae [term 1 LEADER]: Becoming Leader. State: Replica: b302de404f8845cf9f084d646e1b18ae, State: Running, Role: LEADER
I20260924 13:51:34.774057 1807 sys_catalog.cc:564] T 00000000000000000000000000000000 P b302de404f8845cf9f084d646e1b18ae [sys.catalog]: configured and running, proceeding with master startup.
I20260924 13:51:34.774217 1812 consensus_queue.cc:237] T 00000000000000000000000000000000 P b302de404f8845cf9f084d646e1b18ae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b302de404f8845cf9f084d646e1b18ae" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35977 } }
I20260924 13:51:34.774621 1813 sys_catalog.cc:455] T 00000000000000000000000000000000 P b302de404f8845cf9f084d646e1b18ae [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b302de404f8845cf9f084d646e1b18ae" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b302de404f8845cf9f084d646e1b18ae" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35977 } } }
I20260924 13:51:34.774705 1813 sys_catalog.cc:458] T 00000000000000000000000000000000 P b302de404f8845cf9f084d646e1b18ae [sys.catalog]: This master's current role is: LEADER
I20260924 13:51:34.774947 1814 sys_catalog.cc:455] T 00000000000000000000000000000000 P b302de404f8845cf9f084d646e1b18ae [sys.catalog]: SysCatalogTable state changed. Reason: New leader b302de404f8845cf9f084d646e1b18ae. Latest consensus state: current_term: 1 leader_uuid: "b302de404f8845cf9f084d646e1b18ae" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b302de404f8845cf9f084d646e1b18ae" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35977 } } }
I20260924 13:51:34.775079 1814 sys_catalog.cc:458] T 00000000000000000000000000000000 P b302de404f8845cf9f084d646e1b18ae [sys.catalog]: This master's current role is: LEADER
I20260924 13:51:34.775244 1820 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 13:51:34.775790 1820 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Sep 24 13:51:34 dist-test-slave-sjl2 krb5kdc[1726](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257894, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260924 13:51:34.777354 1820 catalog_manager.cc:1347] Generated new cluster ID: ed0dec97706043acbcf335e25ee86734
I20260924 13:51:34.777407 1820 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 13:51:34.781293 1811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:34.769775 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50269 (local address 127.0.0.1:35977)
0924 13:51:34.769930 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0924 13:51:34.769939 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0924 13:51:34.769972 (+ 33us) server_negotiation.cc:407] Connection header received
0924 13:51:34.770528 (+ 556us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:51:34.770537 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:51:34.770601 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:51:34.770815 (+ 214us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:51:34.771420 (+ 605us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:51:34.772777 (+ 1357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:51:34.773654 (+ 877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:51:34.774244 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:51:34.774348 (+ 104us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:51:34.777283 (+ 2935us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:51:34.777288 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:51:34.777288 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 13:51:34.777300 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:51:34.779263 (+ 1963us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:51:34.779727 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:51:34.779730 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:51:34.779730 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:51:34.779828 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:51:34.780048 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:51:34.780049 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:51:34.780049 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:51:34.780327 (+ 278us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:51:34.780476 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0924 13:51:34.780993 (+ 517us) server_negotiation.cc:299] Negotiation successful
0924 13:51:34.781093 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":109,"thread_start_us":81,"threads_started":1}
I20260924 13:51:34.782327 13911 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260924 13:51:34.791078 1811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:34.783934 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39279 (local address 127.0.0.1:35977)
0924 13:51:34.783968 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0924 13:51:34.783969 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0924 13:51:34.783979 (+ 10us) server_negotiation.cc:407] Connection header received
0924 13:51:34.784025 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:51:34.784026 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:51:34.784036 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:51:34.784093 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:51:34.784687 (+ 594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:51:34.785445 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:51:34.786464 (+ 1019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:51:34.786787 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:51:34.786851 (+ 64us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:51:34.788204 (+ 1353us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:51:34.788205 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:51:34.788206 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 13:51:34.788212 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:51:34.789752 (+ 1540us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:51:34.790185 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:51:34.790186 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:51:34.790187 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:51:34.790242 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:51:34.790447 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:51:34.790448 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:51:34.790448 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:51:34.790575 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:51:34.790667 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0924 13:51:34.790809 (+ 142us) server_negotiation.cc:299] Negotiation successful
0924 13:51:34.790871 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260924 13:51:34.808108 1820 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 13:51:34.808650 1820 catalog_manager.cc:1504] Loading token signing keys...
I20260924 13:51:34.820377 1820 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b302de404f8845cf9f084d646e1b18ae: Generated new TSK 0
I20260924 13:51:34.820616 1820 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260924 13:51:34.822646 13911 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJN9WRF/build/release/bin/kudu with pid 1739
W20260924 13:51:34.826797 1724 connection.cc:535] client connection to 127.0.0.1:35977 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:35977): Connection reset by peer (error 104)
2026-09-24T13:51:34Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1240 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790257629041645-13911-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 13:51:34 dist-test-slave-sjl2 krb5kdc[1838](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 13:51:34 dist-test-slave-sjl2 krb5kdc[1838](info): set up 2 sockets
Sep 24 13:51:34 dist-test-slave-sjl2 krb5kdc[1838](info): commencing operation
krb5kdc: starting...
W20260924 13:51:35.789987 13911 mini_kdc.cc:121] Time spent starting KDC: real 0.950s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790257629041645-13911-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790257629041645-13911-0/krb5kdc/test-user.keytab.
Sep 24 13:51:35 dist-test-slave-sjl2 krb5kdc[1838](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257895, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T13:51:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T13:51:35Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790257629041645-13911-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790257629041645-13911-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260924 13:51:35.912734 13911 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJN9WRF/build/release/bin/kudu
/tmp/dist-test-taskJN9WRF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790257629041645-13911-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790257629041645-13911-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790257629041645-13911-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790257629041645-13911-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42271
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790257629041645-13911-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38289
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42271
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790257629041645-13911-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790257629041645-13911-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790257629041645-13911-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 13:51:35.991811 1851 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 13:51:35.992018 1851 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 13:51:35.992049 1851 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 13:51:35.993266 1851 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 13:51:35.993310 1851 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 13:51:35.993328 1851 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 13:51:35.993345 1851 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 13:51:35.993359 1851 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 13:51:35.994822 1851 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38289
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790257629041645-13911-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790257629041645-13911-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42271
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790257629041645-13911-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42271
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790257629041645-13911-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1851
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790257629041645-13911-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 12:29:37 UTC on ccycloud.quasar-iqzbxn.root.comops.site
build id 4670106
I20260924 13:51:35.995050 1851 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 13:51:35.995263 1851 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 13:51:35.999959 1857 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 13:51:35.999951 1856 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 13:51:36.000058 1851 server_base.cc:1034] running on GCE node
W20260924 13:51:35.999951 1859 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 13:51:36.000413 1851 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 13:51:36.000694 1851 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 13:51:36.001844 1851 hybrid_clock.cc:648] HybridClock initialized: now 1790257896001822 us; error 32 us; skew 500 ppm
Sep 24 13:51:35 dist-test-slave-sjl2 krb5kdc[1838](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257895, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 13:51:36.004383 1851 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260924 13:51:36.004803 1851 webserver.cc:460] Webserver started at http://127.0.0.1:45163/ using document root <none> and password file <none>
I20260924 13:51:36.005039 1851 fs_manager.cc:359] Metadata directory not provided
I20260924 13:51:36.005085 1851 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 13:51:36.005182 1851 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 13:51:36.006207 1851 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790257629041645-13911-0/minicluster-data/master-0/data/instance:
uuid: "f7e4d6e03fd2416daa3a2a667b345ecb"
format_stamp: "Formatted at 2026-09-24 13:51:36 on dist-test-slave-sjl2"
server_key: "7700874312b1bde649b359bdd462a9c7"
server_key_iv: "58a7df81085727457166962709488982"
server_key_version: "encryptionkey@0"
I20260924 13:51:36.006515 1851 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790257629041645-13911-0/minicluster-data/master-0/wal/instance:
uuid: "f7e4d6e03fd2416daa3a2a667b345ecb"
format_stamp: "Formatted at 2026-09-24 13:51:36 on dist-test-slave-sjl2"
server_key: "7700874312b1bde649b359bdd462a9c7"
server_key_iv: "58a7df81085727457166962709488982"
server_key_version: "encryptionkey@0"
I20260924 13:51:36.007747 1851 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260924 13:51:36.008414 1866 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 13:51:36.008580 1851 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 13:51:36.008666 1851 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790257629041645-13911-0/minicluster-data/master-0/data,/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790257629041645-13911-0/minicluster-data/master-0/wal
uuid: "f7e4d6e03fd2416daa3a2a667b345ecb"
format_stamp: "Formatted at 2026-09-24 13:51:36 on dist-test-slave-sjl2"
server_key: "7700874312b1bde649b359bdd462a9c7"
server_key_iv: "58a7df81085727457166962709488982"
server_key_version: "encryptionkey@0"
I20260924 13:51:36.008723 1851 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790257629041645-13911-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790257629041645-13911-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790257629041645-13911-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 13:51:36.022564 1851 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 13:51:36.024775 1851 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 13:51:36.024891 1851 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 13:51:36.028683 1851 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42271
I20260924 13:51:36.028730 1918 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42271 every 8 connection(s)
I20260924 13:51:36.029062 1851 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790257629041645-13911-0/minicluster-data/master-0/data/info.pb
I20260924 13:51:36.029960 1919 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 13:51:36.032382 1919 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f7e4d6e03fd2416daa3a2a667b345ecb: Bootstrap starting.
I20260924 13:51:36.032984 1919 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f7e4d6e03fd2416daa3a2a667b345ecb: Neither blocks nor log segments found. Creating new log.
I20260924 13:51:36.033255 1919 log.cc:824] T 00000000000000000000000000000000 P f7e4d6e03fd2416daa3a2a667b345ecb: Log is configured to *not* fsync() on all Append() calls
I20260924 13:51:36.033952 1919 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f7e4d6e03fd2416daa3a2a667b345ecb: No bootstrap required, opened a new log
I20260924 13:51:36.035549 1919 raft_consensus.cc:348] T 00000000000000000000000000000000 P f7e4d6e03fd2416daa3a2a667b345ecb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f7e4d6e03fd2416daa3a2a667b345ecb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42271 } }
I20260924 13:51:36.035718 1919 raft_consensus.cc:374] T 00000000000000000000000000000000 P f7e4d6e03fd2416daa3a2a667b345ecb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 13:51:36.035753 1919 raft_consensus.cc:729] T 00000000000000000000000000000000 P f7e4d6e03fd2416daa3a2a667b345ecb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f7e4d6e03fd2416daa3a2a667b345ecb, State: Initialized, Role: FOLLOWER
I20260924 13:51:36.035890 1919 consensus_queue.cc:260] T 00000000000000000000000000000000 P f7e4d6e03fd2416daa3a2a667b345ecb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f7e4d6e03fd2416daa3a2a667b345ecb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42271 } }
I20260924 13:51:36.035959 1919 raft_consensus.cc:388] T 00000000000000000000000000000000 P f7e4d6e03fd2416daa3a2a667b345ecb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 13:51:36.035991 1919 raft_consensus.cc:482] T 00000000000000000000000000000000 P f7e4d6e03fd2416daa3a2a667b345ecb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 13:51:36.036042 1919 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f7e4d6e03fd2416daa3a2a667b345ecb [term 0 FOLLOWER]: Advancing to term 1
I20260924 13:51:36.036584 1919 raft_consensus.cc:504] T 00000000000000000000000000000000 P f7e4d6e03fd2416daa3a2a667b345ecb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f7e4d6e03fd2416daa3a2a667b345ecb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42271 } }
I20260924 13:51:36.036684 1919 leader_election.cc:302] T 00000000000000000000000000000000 P f7e4d6e03fd2416daa3a2a667b345ecb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f7e4d6e03fd2416daa3a2a667b345ecb; no voters:
I20260924 13:51:36.036859 1919 leader_election.cc:288] T 00000000000000000000000000000000 P f7e4d6e03fd2416daa3a2a667b345ecb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 13:51:36.036903 1922 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f7e4d6e03fd2416daa3a2a667b345ecb [term 1 FOLLOWER]: Leader election won for term 1
I20260924 13:51:36.037081 1922 raft_consensus.cc:686] T 00000000000000000000000000000000 P f7e4d6e03fd2416daa3a2a667b345ecb [term 1 LEADER]: Becoming Leader. State: Replica: f7e4d6e03fd2416daa3a2a667b345ecb, State: Running, Role: LEADER
I20260924 13:51:36.037130 1919 sys_catalog.cc:564] T 00000000000000000000000000000000 P f7e4d6e03fd2416daa3a2a667b345ecb [sys.catalog]: configured and running, proceeding with master startup.
I20260924 13:51:36.037168 1922 consensus_queue.cc:237] T 00000000000000000000000000000000 P f7e4d6e03fd2416daa3a2a667b345ecb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f7e4d6e03fd2416daa3a2a667b345ecb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42271 } }
I20260924 13:51:36.037524 13911 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJN9WRF/build/release/bin/kudu as pid 1851
I20260924 13:51:36.037654 13911 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790257629041645-13911-0/minicluster-data/master-0/wal/instance
I20260924 13:51:36.037894 13911 external_mini_cluster.cc:1428] Setting key 5d2aad69389b97cc63997397fe4883ed
I20260924 13:51:36.038015 1924 sys_catalog.cc:455] T 00000000000000000000000000000000 P f7e4d6e03fd2416daa3a2a667b345ecb [sys.catalog]: SysCatalogTable state changed. Reason: New leader f7e4d6e03fd2416daa3a2a667b345ecb. Latest consensus state: current_term: 1 leader_uuid: "f7e4d6e03fd2416daa3a2a667b345ecb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f7e4d6e03fd2416daa3a2a667b345ecb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42271 } } }
I20260924 13:51:36.038089 1924 sys_catalog.cc:458] T 00000000000000000000000000000000 P f7e4d6e03fd2416daa3a2a667b345ecb [sys.catalog]: This master's current role is: LEADER
I20260924 13:51:36.038360 1932 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 13:51:36.038477 1923 sys_catalog.cc:455] T 00000000000000000000000000000000 P f7e4d6e03fd2416daa3a2a667b345ecb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f7e4d6e03fd2416daa3a2a667b345ecb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f7e4d6e03fd2416daa3a2a667b345ecb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42271 } } }
I20260924 13:51:36.038547 1923 sys_catalog.cc:458] T 00000000000000000000000000000000 P f7e4d6e03fd2416daa3a2a667b345ecb [sys.catalog]: This master's current role is: LEADER
I20260924 13:51:36.038873 1932 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 13:51:36.040482 1932 catalog_manager.cc:1347] Generated new cluster ID: f4631a635a9d42509ccaddd9d401721e
I20260924 13:51:36.040534 1932 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 13:51:36.045279 1939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:36.038567 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35795 (local address 127.0.0.1:42271)
0924 13:51:36.038727 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0924 13:51:36.038737 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0924 13:51:36.039652 (+ 915us) server_negotiation.cc:407] Connection header received
0924 13:51:36.039802 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:51:36.039810 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:51:36.039934 (+ 124us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:51:36.040072 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:51:36.040313 (+ 241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:51:36.041801 (+ 1488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:51:36.042898 (+ 1097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:51:36.043526 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:51:36.043634 (+ 108us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:51:36.044358 (+ 724us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:51:36.044363 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:51:36.044363 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 13:51:36.044571 (+ 208us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:51:36.044610 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 13:51:36.044625 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:51:36.044726 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0924 13:51:36.044809 (+ 83us) server_negotiation.cc:299] Negotiation successful
0924 13:51:36.044839 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":107,"thread_start_us":86,"threads_started":1}
I20260924 13:51:36.046263 13911 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260924 13:51:36.050949 1932 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 13:51:36.051595 1932 catalog_manager.cc:1504] Loading token signing keys...
I20260924 13:51:36.052472 1939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:36.047866 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53815 (local address 127.0.0.1:42271)
0924 13:51:36.047917 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0924 13:51:36.047918 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0924 13:51:36.049084 (+ 1166us) server_negotiation.cc:407] Connection header received
0924 13:51:36.049156 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:51:36.049157 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:51:36.049168 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:51:36.049230 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:51:36.049361 (+ 131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:51:36.050130 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:51:36.050996 (+ 866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:51:36.051321 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:51:36.051394 (+ 73us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:51:36.052113 (+ 719us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:51:36.052114 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:51:36.052115 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 13:51:36.052120 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:51:36.052138 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 13:51:36.052142 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:51:36.052210 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0924 13:51:36.052278 (+ 68us) server_negotiation.cc:299] Negotiation successful
0924 13:51:36.052296 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260924 13:51:36.056226 1932 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f7e4d6e03fd2416daa3a2a667b345ecb: Generated new TSK 0
I20260924 13:51:36.056429 1932 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260924 13:51:36.061102 13911 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJN9WRF/build/release/bin/kudu with pid 1851
W20260924 13:51:36.064553 1836 connection.cc:535] client connection to 127.0.0.1:42271 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:42271): Connection reset by peer (error 104)
2026-09-24T13:51:36Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1237 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790257629041645-13911-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 13:51:36 dist-test-slave-sjl2 krb5kdc[1950](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 13:51:36 dist-test-slave-sjl2 krb5kdc[1950](info): set up 2 sockets
Sep 24 13:51:36 dist-test-slave-sjl2 krb5kdc[1950](info): commencing operation
krb5kdc: starting...
W20260924 13:51:37.013813 13911 mini_kdc.cc:121] Time spent starting KDC: real 0.936s user 0.005s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790257629041645-13911-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790257629041645-13911-0/krb5kdc/test-user.keytab.
Sep 24 13:51:37 dist-test-slave-sjl2 krb5kdc[1950](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257897, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T13:51:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T13:51:37Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790257629041645-13911-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790257629041645-13911-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260924 13:51:37.132040 13911 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJN9WRF/build/release/bin/kudu
/tmp/dist-test-taskJN9WRF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790257629041645-13911-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790257629041645-13911-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790257629041645-13911-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790257629041645-13911-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44961
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790257629041645-13911-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39847
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44961
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790257629041645-13911-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790257629041645-13911-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790257629041645-13911-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 13:51:37.211333 1963 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 13:51:37.211539 1963 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 13:51:37.211560 1963 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 13:51:37.212695 1963 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 13:51:37.212739 1963 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 13:51:37.212752 1963 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 13:51:37.212774 1963 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 13:51:37.212796 1963 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 13:51:37.214288 1963 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39847
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790257629041645-13911-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790257629041645-13911-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44961
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790257629041645-13911-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44961
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790257629041645-13911-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1963
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790257629041645-13911-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 12:29:37 UTC on ccycloud.quasar-iqzbxn.root.comops.site
build id 4670106
I20260924 13:51:37.214521 1963 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 13:51:37.214723 1963 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 13:51:37.219515 1971 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 13:51:37.219524 1963 server_base.cc:1034] running on GCE node
W20260924 13:51:37.219563 1968 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 13:51:37.219760 1969 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 13:51:37.220014 1963 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 13:51:37.220269 1963 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 13:51:37.221400 1963 hybrid_clock.cc:648] HybridClock initialized: now 1790257897221386 us; error 32 us; skew 500 ppm
Sep 24 13:51:37 dist-test-slave-sjl2 krb5kdc[1950](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257897, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 13:51:37.223682 1963 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260924 13:51:37.224089 1963 webserver.cc:460] Webserver started at http://127.0.0.1:40467/ using document root <none> and password file <none>
I20260924 13:51:37.224303 1963 fs_manager.cc:359] Metadata directory not provided
I20260924 13:51:37.224350 1963 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 13:51:37.224471 1963 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 13:51:37.225574 1963 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790257629041645-13911-0/minicluster-data/master-0/data/instance:
uuid: "7b879f13a77248159366f25425808c7c"
format_stamp: "Formatted at 2026-09-24 13:51:37 on dist-test-slave-sjl2"
server_key: "46fe5dae0bf7b2dd7fe82a6d0c6686a2"
server_key_iv: "8a205f792e569754465a4719657d9e8c"
server_key_version: "encryptionkey@0"
I20260924 13:51:37.225912 1963 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790257629041645-13911-0/minicluster-data/master-0/wal/instance:
uuid: "7b879f13a77248159366f25425808c7c"
format_stamp: "Formatted at 2026-09-24 13:51:37 on dist-test-slave-sjl2"
server_key: "46fe5dae0bf7b2dd7fe82a6d0c6686a2"
server_key_iv: "8a205f792e569754465a4719657d9e8c"
server_key_version: "encryptionkey@0"
I20260924 13:51:37.227185 1963 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260924 13:51:37.227945 1978 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 13:51:37.228117 1963 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 13:51:37.228199 1963 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790257629041645-13911-0/minicluster-data/master-0/data,/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790257629041645-13911-0/minicluster-data/master-0/wal
uuid: "7b879f13a77248159366f25425808c7c"
format_stamp: "Formatted at 2026-09-24 13:51:37 on dist-test-slave-sjl2"
server_key: "46fe5dae0bf7b2dd7fe82a6d0c6686a2"
server_key_iv: "8a205f792e569754465a4719657d9e8c"
server_key_version: "encryptionkey@0"
I20260924 13:51:37.228288 1963 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790257629041645-13911-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790257629041645-13911-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790257629041645-13911-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 13:51:37.231575 1963 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 13:51:37.233448 1963 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 13:51:37.233575 1963 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 13:51:37.237267 1963 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44961
I20260924 13:51:37.237287 2030 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44961 every 8 connection(s)
I20260924 13:51:37.237622 1963 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790257629041645-13911-0/minicluster-data/master-0/data/info.pb
I20260924 13:51:37.238514 2031 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 13:51:37.240883 2031 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7b879f13a77248159366f25425808c7c: Bootstrap starting.
I20260924 13:51:37.241529 2031 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7b879f13a77248159366f25425808c7c: Neither blocks nor log segments found. Creating new log.
I20260924 13:51:37.241797 2031 log.cc:824] T 00000000000000000000000000000000 P 7b879f13a77248159366f25425808c7c: Log is configured to *not* fsync() on all Append() calls
I20260924 13:51:37.242597 2031 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7b879f13a77248159366f25425808c7c: No bootstrap required, opened a new log
I20260924 13:51:37.244266 2031 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7b879f13a77248159366f25425808c7c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b879f13a77248159366f25425808c7c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44961 } }
I20260924 13:51:37.244422 2031 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7b879f13a77248159366f25425808c7c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 13:51:37.244444 2031 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7b879f13a77248159366f25425808c7c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7b879f13a77248159366f25425808c7c, State: Initialized, Role: FOLLOWER
I20260924 13:51:37.244588 2031 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7b879f13a77248159366f25425808c7c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b879f13a77248159366f25425808c7c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44961 } }
I20260924 13:51:37.244655 2031 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7b879f13a77248159366f25425808c7c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 13:51:37.244676 2031 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7b879f13a77248159366f25425808c7c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 13:51:37.244724 2031 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7b879f13a77248159366f25425808c7c [term 0 FOLLOWER]: Advancing to term 1
I20260924 13:51:37.245384 2031 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7b879f13a77248159366f25425808c7c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b879f13a77248159366f25425808c7c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44961 } }
I20260924 13:51:37.245481 2031 leader_election.cc:302] T 00000000000000000000000000000000 P 7b879f13a77248159366f25425808c7c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7b879f13a77248159366f25425808c7c; no voters:
I20260924 13:51:37.245671 2031 leader_election.cc:288] T 00000000000000000000000000000000 P 7b879f13a77248159366f25425808c7c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 13:51:37.245734 2034 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7b879f13a77248159366f25425808c7c [term 1 FOLLOWER]: Leader election won for term 1
I20260924 13:51:37.245908 2031 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7b879f13a77248159366f25425808c7c [sys.catalog]: configured and running, proceeding with master startup.
I20260924 13:51:37.245914 2034 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7b879f13a77248159366f25425808c7c [term 1 LEADER]: Becoming Leader. State: Replica: 7b879f13a77248159366f25425808c7c, State: Running, Role: LEADER
I20260924 13:51:37.246035 2034 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7b879f13a77248159366f25425808c7c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b879f13a77248159366f25425808c7c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44961 } }
I20260924 13:51:37.246560 13911 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJN9WRF/build/release/bin/kudu as pid 1963
I20260924 13:51:37.246692 13911 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790257629041645-13911-0/minicluster-data/master-0/wal/instance
I20260924 13:51:37.246888 13911 external_mini_cluster.cc:1428] Setting key 6cd4778421dd98f755c20047264cac88
I20260924 13:51:37.247255 2035 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7b879f13a77248159366f25425808c7c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7b879f13a77248159366f25425808c7c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b879f13a77248159366f25425808c7c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44961 } } }
I20260924 13:51:37.247344 2035 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7b879f13a77248159366f25425808c7c [sys.catalog]: This master's current role is: LEADER
I20260924 13:51:37.247232 2036 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7b879f13a77248159366f25425808c7c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7b879f13a77248159366f25425808c7c. Latest consensus state: current_term: 1 leader_uuid: "7b879f13a77248159366f25425808c7c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b879f13a77248159366f25425808c7c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44961 } } }
I20260924 13:51:37.247560 2036 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7b879f13a77248159366f25425808c7c [sys.catalog]: This master's current role is: LEADER
I20260924 13:51:37.247836 2050 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 13:51:37.248390 2050 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 13:51:37.249434 2051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:37.247407 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36463 (local address 127.0.0.1:44961)
0924 13:51:37.247540 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0924 13:51:37.247551 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0924 13:51:37.247598 (+ 47us) server_negotiation.cc:407] Connection header received
0924 13:51:37.247733 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:51:37.247740 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:51:37.247774 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:51:37.247933 (+ 159us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:51:37.248988 (+ 1055us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:51:37.248992 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:51:37.248992 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 13:51:37.248997 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:51:37.249043 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 13:51:37.249058 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:51:37.249129 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0924 13:51:37.249216 (+ 87us) server_negotiation.cc:299] Negotiation successful
0924 13:51:37.249258 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78,"thread_start_us":54,"threads_started":1}
I20260924 13:51:37.250108 2050 catalog_manager.cc:1347] Generated new cluster ID: 690e470f3eea4e00a958127430056cd4
I20260924 13:51:37.250155 2050 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 13:51:37.250639 13911 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260924 13:51:37.258088 2051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:37.253183 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40115 (local address 127.0.0.1:44961)
0924 13:51:37.253228 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0924 13:51:37.253230 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:51:37.257107 (+ 3877us) server_negotiation.cc:407] Connection header received
0924 13:51:37.257172 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:51:37.257173 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:51:37.257180 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:51:37.257255 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:51:37.257855 (+ 600us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:51:37.257857 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:51:37.257857 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 13:51:37.257862 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:51:37.257881 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 13:51:37.257886 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:51:37.257939 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0924 13:51:37.257993 (+ 54us) server_negotiation.cc:299] Negotiation successful
0924 13:51:37.258007 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260924 13:51:37.283883 2050 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 13:51:37.284082 2050 catalog_manager.cc:1504] Loading token signing keys...
I20260924 13:51:37.290148 2050 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7b879f13a77248159366f25425808c7c: Generated new TSK 0
I20260924 13:51:37.290369 2050 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260924 13:51:37.301374 13911 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJN9WRF/build/release/bin/kudu with pid 1963
2026-09-24T13:51:37Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1240 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790257629041645-13911-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 13:51:37 dist-test-slave-sjl2 krb5kdc[2063](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 13:51:37 dist-test-slave-sjl2 krb5kdc[2063](info): set up 2 sockets
Sep 24 13:51:37 dist-test-slave-sjl2 krb5kdc[2063](info): commencing operation
krb5kdc: starting...
W20260924 13:51:38.249934 13911 mini_kdc.cc:121] Time spent starting KDC: real 0.932s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790257629041645-13911-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790257629041645-13911-0/krb5kdc/test-user.keytab.
Sep 24 13:51:38 dist-test-slave-sjl2 krb5kdc[2063](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257898, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T13:51:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T13:51:38Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790257629041645-13911-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790257629041645-13911-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260924 13:51:38.367487 13911 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJN9WRF/build/release/bin/kudu
/tmp/dist-test-taskJN9WRF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790257629041645-13911-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790257629041645-13911-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790257629041645-13911-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790257629041645-13911-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45413
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790257629041645-13911-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42905
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45413
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790257629041645-13911-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790257629041645-13911-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790257629041645-13911-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 13:51:38.446614 2076 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 13:51:38.446851 2076 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 13:51:38.446874 2076 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 13:51:38.448091 2076 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 13:51:38.448146 2076 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 13:51:38.448160 2076 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 13:51:38.448171 2076 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 13:51:38.448182 2076 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 13:51:38.449764 2076 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42905
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790257629041645-13911-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790257629041645-13911-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45413
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790257629041645-13911-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45413
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790257629041645-13911-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2076
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790257629041645-13911-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 12:29:37 UTC on ccycloud.quasar-iqzbxn.root.comops.site
build id 4670106
I20260924 13:51:38.450019 2076 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 13:51:38.450258 2076 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 13:51:38.455050 2081 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 13:51:38.455108 2076 server_base.cc:1034] running on GCE node
W20260924 13:51:38.455039 2082 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 13:51:38.455039 2084 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 13:51:38.455504 2076 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 13:51:38.455762 2076 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 13:51:38.456951 2076 hybrid_clock.cc:648] HybridClock initialized: now 1790257898456930 us; error 28 us; skew 500 ppm
Sep 24 13:51:38 dist-test-slave-sjl2 krb5kdc[2063](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257898, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 13:51:38.463459 2076 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260924 13:51:38.463883 2076 webserver.cc:460] Webserver started at http://127.0.0.1:34121/ using document root <none> and password file <none>
I20260924 13:51:38.464067 2076 fs_manager.cc:359] Metadata directory not provided
I20260924 13:51:38.464107 2076 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 13:51:38.464212 2076 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 13:51:38.465299 2076 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790257629041645-13911-0/minicluster-data/master-0/data/instance:
uuid: "0b7fc21d6b7242678410bd6466a1f385"
format_stamp: "Formatted at 2026-09-24 13:51:38 on dist-test-slave-sjl2"
server_key: "ed3725a76d56671c21b80f1dcffef756"
server_key_iv: "61c6b26e07bea3640b31386cf2ec2288"
server_key_version: "encryptionkey@0"
I20260924 13:51:38.465602 2076 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790257629041645-13911-0/minicluster-data/master-0/wal/instance:
uuid: "0b7fc21d6b7242678410bd6466a1f385"
format_stamp: "Formatted at 2026-09-24 13:51:38 on dist-test-slave-sjl2"
server_key: "ed3725a76d56671c21b80f1dcffef756"
server_key_iv: "61c6b26e07bea3640b31386cf2ec2288"
server_key_version: "encryptionkey@0"
I20260924 13:51:38.466979 2076 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260924 13:51:38.467804 2091 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 13:51:38.468001 2076 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260924 13:51:38.468087 2076 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790257629041645-13911-0/minicluster-data/master-0/data,/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790257629041645-13911-0/minicluster-data/master-0/wal
uuid: "0b7fc21d6b7242678410bd6466a1f385"
format_stamp: "Formatted at 2026-09-24 13:51:38 on dist-test-slave-sjl2"
server_key: "ed3725a76d56671c21b80f1dcffef756"
server_key_iv: "61c6b26e07bea3640b31386cf2ec2288"
server_key_version: "encryptionkey@0"
I20260924 13:51:38.468149 2076 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790257629041645-13911-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790257629041645-13911-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790257629041645-13911-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 13:51:38.474584 2076 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 13:51:38.476348 2076 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 13:51:38.476454 2076 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 13:51:38.480118 2076 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45413
I20260924 13:51:38.480163 2143 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45413 every 8 connection(s)
I20260924 13:51:38.480475 2076 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790257629041645-13911-0/minicluster-data/master-0/data/info.pb
I20260924 13:51:38.481393 2144 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 13:51:38.482056 13911 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJN9WRF/build/release/bin/kudu as pid 2076
I20260924 13:51:38.482141 13911 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790257629041645-13911-0/minicluster-data/master-0/wal/instance
I20260924 13:51:38.482316 13911 external_mini_cluster.cc:1428] Setting key c71d0f8d477c4d360b922537e5d4dd7c
I20260924 13:51:38.484299 2144 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0b7fc21d6b7242678410bd6466a1f385: Bootstrap starting.
I20260924 13:51:38.484860 2144 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0b7fc21d6b7242678410bd6466a1f385: Neither blocks nor log segments found. Creating new log.
I20260924 13:51:38.485152 2144 log.cc:824] T 00000000000000000000000000000000 P 0b7fc21d6b7242678410bd6466a1f385: Log is configured to *not* fsync() on all Append() calls
I20260924 13:51:38.485764 2144 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0b7fc21d6b7242678410bd6466a1f385: No bootstrap required, opened a new log
I20260924 13:51:38.487591 2144 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0b7fc21d6b7242678410bd6466a1f385 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b7fc21d6b7242678410bd6466a1f385" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45413 } }
I20260924 13:51:38.487725 2144 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0b7fc21d6b7242678410bd6466a1f385 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 13:51:38.487757 2144 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0b7fc21d6b7242678410bd6466a1f385 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0b7fc21d6b7242678410bd6466a1f385, State: Initialized, Role: FOLLOWER
I20260924 13:51:38.487880 2144 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0b7fc21d6b7242678410bd6466a1f385 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b7fc21d6b7242678410bd6466a1f385" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45413 } }
I20260924 13:51:38.487952 2144 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0b7fc21d6b7242678410bd6466a1f385 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 13:51:38.487990 2144 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0b7fc21d6b7242678410bd6466a1f385 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 13:51:38.488034 2144 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0b7fc21d6b7242678410bd6466a1f385 [term 0 FOLLOWER]: Advancing to term 1
Sep 24 13:51:38 dist-test-slave-sjl2 krb5kdc[2063](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257898, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260924 13:51:38.488556 2144 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0b7fc21d6b7242678410bd6466a1f385 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b7fc21d6b7242678410bd6466a1f385" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45413 } }
I20260924 13:51:38.488656 2144 leader_election.cc:302] T 00000000000000000000000000000000 P 0b7fc21d6b7242678410bd6466a1f385 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0b7fc21d6b7242678410bd6466a1f385; no voters:
I20260924 13:51:38.488829 2144 leader_election.cc:288] T 00000000000000000000000000000000 P 0b7fc21d6b7242678410bd6466a1f385 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 13:51:38.488857 2149 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0b7fc21d6b7242678410bd6466a1f385 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 13:51:38.489084 2149 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0b7fc21d6b7242678410bd6466a1f385 [term 1 LEADER]: Becoming Leader. State: Replica: 0b7fc21d6b7242678410bd6466a1f385, State: Running, Role: LEADER
I20260924 13:51:38.489111 2144 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0b7fc21d6b7242678410bd6466a1f385 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 13:51:38.489181 2149 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0b7fc21d6b7242678410bd6466a1f385 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b7fc21d6b7242678410bd6466a1f385" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45413 } }
I20260924 13:51:38.490283 2149 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0b7fc21d6b7242678410bd6466a1f385 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0b7fc21d6b7242678410bd6466a1f385. Latest consensus state: current_term: 1 leader_uuid: "0b7fc21d6b7242678410bd6466a1f385" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b7fc21d6b7242678410bd6466a1f385" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45413 } } }
I20260924 13:51:38.490413 2149 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0b7fc21d6b7242678410bd6466a1f385 [sys.catalog]: This master's current role is: LEADER
I20260924 13:51:38.490520 2150 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0b7fc21d6b7242678410bd6466a1f385 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0b7fc21d6b7242678410bd6466a1f385" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b7fc21d6b7242678410bd6466a1f385" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45413 } } }
I20260924 13:51:38.490691 2150 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0b7fc21d6b7242678410bd6466a1f385 [sys.catalog]: This master's current role is: LEADER
W20260924 13:51:38.490957 2163 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 0b7fc21d6b7242678410bd6466a1f385: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260924 13:51:38.491006 2163 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260924 13:51:38.491181 2165 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 13:51:38.491333 2165 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 13:51:38.492892 2147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:38.482859 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46183 (local address 127.0.0.1:45413)
0924 13:51:38.483038 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0924 13:51:38.483048 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0924 13:51:38.483088 (+ 40us) server_negotiation.cc:407] Connection header received
0924 13:51:38.483568 (+ 480us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:51:38.483577 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:51:38.483642 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:51:38.483787 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:51:38.484443 (+ 656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:51:38.485408 (+ 965us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:51:38.486119 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:51:38.486534 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:51:38.486630 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:51:38.488937 (+ 2307us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:51:38.488942 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:51:38.488942 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 13:51:38.488955 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:51:38.491013 (+ 2058us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:51:38.491465 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:51:38.491468 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:51:38.491469 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:51:38.491576 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:51:38.491811 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:51:38.491812 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:51:38.491812 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:51:38.492130 (+ 318us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:51:38.492282 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0924 13:51:38.492557 (+ 275us) server_negotiation.cc:299] Negotiation successful
0924 13:51:38.492713 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":78,"threads_started":1}
I20260924 13:51:38.493026 2165 catalog_manager.cc:1347] Generated new cluster ID: d94913046dab4c26aeea25bdb62735d6
I20260924 13:51:38.493063 2165 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 13:51:38.493813 13911 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260924 13:51:38.502758 2147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:38.495319 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33021 (local address 127.0.0.1:45413)
0924 13:51:38.495357 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0924 13:51:38.495359 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:51:38.495477 (+ 118us) server_negotiation.cc:407] Connection header received
0924 13:51:38.495581 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:51:38.495582 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:51:38.495592 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:51:38.495624 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:51:38.496372 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:51:38.497123 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:51:38.498096 (+ 973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:51:38.498390 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:51:38.498477 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:51:38.499836 (+ 1359us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:51:38.499838 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:51:38.499838 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 13:51:38.499846 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:51:38.501634 (+ 1788us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:51:38.501982 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:51:38.501983 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:51:38.501983 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:51:38.502050 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:51:38.502229 (+ 179us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:51:38.502230 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:51:38.502230 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:51:38.502371 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:51:38.502472 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0924 13:51:38.502567 (+ 95us) server_negotiation.cc:299] Negotiation successful
0924 13:51:38.502656 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260924 13:51:38.507652 2165 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 13:51:38.508205 2165 catalog_manager.cc:1504] Loading token signing keys...
I20260924 13:51:38.512966 2165 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0b7fc21d6b7242678410bd6466a1f385: Generated new TSK 0
I20260924 13:51:38.513134 2165 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260924 13:51:38.517496 13911 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJN9WRF/build/release/bin/kudu with pid 2076
2026-09-24T13:51:38Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1216 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790257629041645-13911-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 13:51:38 dist-test-slave-sjl2 krb5kdc[2175](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 13:51:38 dist-test-slave-sjl2 krb5kdc[2175](info): set up 2 sockets
Sep 24 13:51:38 dist-test-slave-sjl2 krb5kdc[2175](info): commencing operation
krb5kdc: starting...
W20260924 13:51:39.465934 13911 mini_kdc.cc:121] Time spent starting KDC: real 0.931s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790257629041645-13911-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790257629041645-13911-0/krb5kdc/test-user.keytab.
Sep 24 13:51:39 dist-test-slave-sjl2 krb5kdc[2175](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257899, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T13:51:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T13:51:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790257629041645-13911-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790257629041645-13911-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260924 13:51:39.584847 13911 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJN9WRF/build/release/bin/kudu
/tmp/dist-test-taskJN9WRF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790257629041645-13911-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790257629041645-13911-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790257629041645-13911-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790257629041645-13911-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36125
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790257629041645-13911-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36945
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36125
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790257629041645-13911-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790257629041645-13911-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790257629041645-13911-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 13:51:39.664417 2188 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 13:51:39.664646 2188 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 13:51:39.664680 2188 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 13:51:39.665908 2188 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 13:51:39.665956 2188 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 13:51:39.665973 2188 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 13:51:39.665989 2188 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 13:51:39.666004 2188 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 13:51:39.667567 2188 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36945
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790257629041645-13911-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790257629041645-13911-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36125
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790257629041645-13911-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36125
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790257629041645-13911-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2188
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790257629041645-13911-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 12:29:37 UTC on ccycloud.quasar-iqzbxn.root.comops.site
build id 4670106
I20260924 13:51:39.667845 2188 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 13:51:39.668063 2188 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 13:51:39.672938 2196 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 13:51:39.672945 2194 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 13:51:39.672948 2193 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 13:51:39.672999 2188 server_base.cc:1034] running on GCE node
I20260924 13:51:39.673404 2188 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 13:51:39.673712 2188 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 13:51:39.674872 2188 hybrid_clock.cc:648] HybridClock initialized: now 1790257899674855 us; error 31 us; skew 500 ppm
Sep 24 13:51:39 dist-test-slave-sjl2 krb5kdc[2175](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257899, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 13:51:39.677451 2188 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260924 13:51:39.677877 2188 webserver.cc:460] Webserver started at http://127.0.0.1:42977/ using document root <none> and password file <none>
I20260924 13:51:39.678086 2188 fs_manager.cc:359] Metadata directory not provided
I20260924 13:51:39.678128 2188 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 13:51:39.678218 2188 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 13:51:39.679241 2188 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790257629041645-13911-0/minicluster-data/master-0/data/instance:
uuid: "74c3a5834d2444e9aa23854e0c685792"
format_stamp: "Formatted at 2026-09-24 13:51:39 on dist-test-slave-sjl2"
server_key: "78ad25081dff09fdb5f31d8cbbc17f75"
server_key_iv: "4d72ffa8355491ff5a098956f436e357"
server_key_version: "encryptionkey@0"
I20260924 13:51:39.679538 2188 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790257629041645-13911-0/minicluster-data/master-0/wal/instance:
uuid: "74c3a5834d2444e9aa23854e0c685792"
format_stamp: "Formatted at 2026-09-24 13:51:39 on dist-test-slave-sjl2"
server_key: "78ad25081dff09fdb5f31d8cbbc17f75"
server_key_iv: "4d72ffa8355491ff5a098956f436e357"
server_key_version: "encryptionkey@0"
I20260924 13:51:39.680868 2188 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260924 13:51:39.681617 2203 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 13:51:39.681780 2188 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260924 13:51:39.681857 2188 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790257629041645-13911-0/minicluster-data/master-0/data,/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790257629041645-13911-0/minicluster-data/master-0/wal
uuid: "74c3a5834d2444e9aa23854e0c685792"
format_stamp: "Formatted at 2026-09-24 13:51:39 on dist-test-slave-sjl2"
server_key: "78ad25081dff09fdb5f31d8cbbc17f75"
server_key_iv: "4d72ffa8355491ff5a098956f436e357"
server_key_version: "encryptionkey@0"
I20260924 13:51:39.681913 2188 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790257629041645-13911-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790257629041645-13911-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790257629041645-13911-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 13:51:39.698092 2188 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 13:51:39.699847 2188 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 13:51:39.699958 2188 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 13:51:39.703634 2188 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36125
I20260924 13:51:39.703699 2255 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36125 every 8 connection(s)
I20260924 13:51:39.703953 2188 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790257629041645-13911-0/minicluster-data/master-0/data/info.pb
I20260924 13:51:39.704789 2256 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 13:51:39.707042 2256 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 74c3a5834d2444e9aa23854e0c685792: Bootstrap starting.
I20260924 13:51:39.707626 2256 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 74c3a5834d2444e9aa23854e0c685792: Neither blocks nor log segments found. Creating new log.
I20260924 13:51:39.707880 2256 log.cc:824] T 00000000000000000000000000000000 P 74c3a5834d2444e9aa23854e0c685792: Log is configured to *not* fsync() on all Append() calls
I20260924 13:51:39.708616 2256 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 74c3a5834d2444e9aa23854e0c685792: No bootstrap required, opened a new log
I20260924 13:51:39.709592 13911 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJN9WRF/build/release/bin/kudu as pid 2188
I20260924 13:51:39.709722 13911 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790257629041645-13911-0/minicluster-data/master-0/wal/instance
I20260924 13:51:39.709889 13911 external_mini_cluster.cc:1428] Setting key 52870f2237d523d79fd937a691eb555f
I20260924 13:51:39.710680 2256 raft_consensus.cc:348] T 00000000000000000000000000000000 P 74c3a5834d2444e9aa23854e0c685792 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74c3a5834d2444e9aa23854e0c685792" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36125 } }
I20260924 13:51:39.710820 2256 raft_consensus.cc:374] T 00000000000000000000000000000000 P 74c3a5834d2444e9aa23854e0c685792 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 13:51:39.710850 2256 raft_consensus.cc:729] T 00000000000000000000000000000000 P 74c3a5834d2444e9aa23854e0c685792 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 74c3a5834d2444e9aa23854e0c685792, State: Initialized, Role: FOLLOWER
I20260924 13:51:39.711010 2256 consensus_queue.cc:260] T 00000000000000000000000000000000 P 74c3a5834d2444e9aa23854e0c685792 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74c3a5834d2444e9aa23854e0c685792" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36125 } }
I20260924 13:51:39.711097 2256 raft_consensus.cc:388] T 00000000000000000000000000000000 P 74c3a5834d2444e9aa23854e0c685792 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 13:51:39.711122 2256 raft_consensus.cc:482] T 00000000000000000000000000000000 P 74c3a5834d2444e9aa23854e0c685792 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 13:51:39.711181 2256 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 74c3a5834d2444e9aa23854e0c685792 [term 0 FOLLOWER]: Advancing to term 1
I20260924 13:51:39.711699 2256 raft_consensus.cc:504] T 00000000000000000000000000000000 P 74c3a5834d2444e9aa23854e0c685792 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74c3a5834d2444e9aa23854e0c685792" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36125 } }
I20260924 13:51:39.711784 2256 leader_election.cc:302] T 00000000000000000000000000000000 P 74c3a5834d2444e9aa23854e0c685792 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 74c3a5834d2444e9aa23854e0c685792; no voters:
I20260924 13:51:39.711933 2256 leader_election.cc:288] T 00000000000000000000000000000000 P 74c3a5834d2444e9aa23854e0c685792 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 13:51:39.711972 2261 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 74c3a5834d2444e9aa23854e0c685792 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 13:51:39.712121 2261 raft_consensus.cc:686] T 00000000000000000000000000000000 P 74c3a5834d2444e9aa23854e0c685792 [term 1 LEADER]: Becoming Leader. State: Replica: 74c3a5834d2444e9aa23854e0c685792, State: Running, Role: LEADER
I20260924 13:51:39.712196 2256 sys_catalog.cc:564] T 00000000000000000000000000000000 P 74c3a5834d2444e9aa23854e0c685792 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 13:51:39.712283 2261 consensus_queue.cc:237] T 00000000000000000000000000000000 P 74c3a5834d2444e9aa23854e0c685792 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74c3a5834d2444e9aa23854e0c685792" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36125 } }
I20260924 13:51:39.713020 2263 sys_catalog.cc:455] T 00000000000000000000000000000000 P 74c3a5834d2444e9aa23854e0c685792 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "74c3a5834d2444e9aa23854e0c685792" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74c3a5834d2444e9aa23854e0c685792" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36125 } } }
I20260924 13:51:39.713198 2263 sys_catalog.cc:458] T 00000000000000000000000000000000 P 74c3a5834d2444e9aa23854e0c685792 [sys.catalog]: This master's current role is: LEADER
I20260924 13:51:39.713495 2262 sys_catalog.cc:455] T 00000000000000000000000000000000 P 74c3a5834d2444e9aa23854e0c685792 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 74c3a5834d2444e9aa23854e0c685792. Latest consensus state: current_term: 1 leader_uuid: "74c3a5834d2444e9aa23854e0c685792" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74c3a5834d2444e9aa23854e0c685792" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36125 } } }
I20260924 13:51:39.713786 2262 sys_catalog.cc:458] T 00000000000000000000000000000000 P 74c3a5834d2444e9aa23854e0c685792 [sys.catalog]: This master's current role is: LEADER
W20260924 13:51:39.713928 2276 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 74c3a5834d2444e9aa23854e0c685792: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260924 13:51:39.713971 2276 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260924 13:51:39.714056 2277 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 13:51:39.714246 2277 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 13:51:39.715341 2260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:39.710405 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50239 (local address 127.0.0.1:36125)
0924 13:51:39.710576 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0924 13:51:39.710583 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0924 13:51:39.710607 (+ 24us) server_negotiation.cc:407] Connection header received
0924 13:51:39.710702 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:51:39.710708 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:51:39.710749 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:51:39.710873 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:51:39.711212 (+ 339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:51:39.712244 (+ 1032us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:51:39.713185 (+ 941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:51:39.713791 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:51:39.713895 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:51:39.714645 (+ 750us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:51:39.714649 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:51:39.714649 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 13:51:39.714853 (+ 204us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:51:39.714890 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 13:51:39.714904 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:51:39.714973 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0924 13:51:39.715058 (+ 85us) server_negotiation.cc:299] Negotiation successful
0924 13:51:39.715134 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125,"thread_start_us":75,"threads_started":1}
I20260924 13:51:39.715984 2277 catalog_manager.cc:1347] Generated new cluster ID: 6799b087a43944708f2b4b4abb2506a2
I20260924 13:51:39.716033 2277 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 13:51:39.716259 13911 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260924 13:51:39.721436 2260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:39.717787 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33053 (local address 127.0.0.1:36125)
0924 13:51:39.717824 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0924 13:51:39.717826 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:51:39.717907 (+ 81us) server_negotiation.cc:407] Connection header received
0924 13:51:39.718001 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:51:39.718001 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:51:39.718011 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:51:39.718039 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:51:39.718245 (+ 206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:51:39.718944 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:51:39.719993 (+ 1049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:51:39.720313 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:51:39.720353 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:51:39.721164 (+ 811us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:51:39.721165 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:51:39.721166 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 13:51:39.721171 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:51:39.721185 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 13:51:39.721189 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:51:39.721216 (+ 27us) server_negotiation.cc:1035] Waiting for connection context
0924 13:51:39.721317 (+ 101us) server_negotiation.cc:299] Negotiation successful
0924 13:51:39.721342 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260924 13:51:39.735553 2277 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 13:51:39.736099 2277 catalog_manager.cc:1504] Loading token signing keys...
I20260924 13:51:39.742102 2277 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 74c3a5834d2444e9aa23854e0c685792: Generated new TSK 0
I20260924 13:51:39.742321 2277 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260924 13:51:39.745921 13911 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJN9WRF/build/release/bin/kudu with pid 2188
2026-09-24T13:51:39Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1228 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790257629041645-13911-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 13:51:39 dist-test-slave-sjl2 krb5kdc[2287](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 13:51:39 dist-test-slave-sjl2 krb5kdc[2287](info): set up 2 sockets
Sep 24 13:51:39 dist-test-slave-sjl2 krb5kdc[2287](info): commencing operation
krb5kdc: starting...
W20260924 13:51:40.693889 13911 mini_kdc.cc:121] Time spent starting KDC: real 0.930s user 0.001s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790257629041645-13911-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790257629041645-13911-0/krb5kdc/test-user.keytab.
Sep 24 13:51:40 dist-test-slave-sjl2 krb5kdc[2287](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257900, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T13:51:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T13:51:40Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790257629041645-13911-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790257629041645-13911-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260924 13:51:40.811960 13911 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJN9WRF/build/release/bin/kudu
/tmp/dist-test-taskJN9WRF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790257629041645-13911-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790257629041645-13911-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790257629041645-13911-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790257629041645-13911-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35199
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790257629041645-13911-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35167
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35199
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790257629041645-13911-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790257629041645-13911-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790257629041645-13911-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 13:51:40.891017 2300 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 13:51:40.891260 2300 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 13:51:40.891295 2300 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 13:51:40.892504 2300 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 13:51:40.892554 2300 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 13:51:40.892572 2300 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 13:51:40.892588 2300 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 13:51:40.892603 2300 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 13:51:40.894158 2300 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35167
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790257629041645-13911-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790257629041645-13911-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35199
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790257629041645-13911-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35199
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790257629041645-13911-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2300
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790257629041645-13911-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 12:29:37 UTC on ccycloud.quasar-iqzbxn.root.comops.site
build id 4670106
I20260924 13:51:40.894433 2300 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 13:51:40.894658 2300 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260924 13:51:40.899621 2300 server_base.cc:1034] running on GCE node
W20260924 13:51:40.899600 2306 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 13:51:40.899601 2305 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 13:51:40.899603 2308 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 13:51:40.900079 2300 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 13:51:40.900434 2300 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 13:51:40.901603 2300 hybrid_clock.cc:648] HybridClock initialized: now 1790257900901587 us; error 28 us; skew 500 ppm
Sep 24 13:51:40 dist-test-slave-sjl2 krb5kdc[2287](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257900, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 13:51:40.904085 2300 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260924 13:51:40.904556 2300 webserver.cc:460] Webserver started at http://127.0.0.1:42549/ using document root <none> and password file <none>
I20260924 13:51:40.904760 2300 fs_manager.cc:359] Metadata directory not provided
I20260924 13:51:40.904806 2300 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 13:51:40.904947 2300 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 13:51:40.906013 2300 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790257629041645-13911-0/minicluster-data/master-0/data/instance:
uuid: "9d1bd9ffa31948908abf5bd659b4bce6"
format_stamp: "Formatted at 2026-09-24 13:51:40 on dist-test-slave-sjl2"
server_key: "f1f0ee94a7e9ee2e824bbd6c74867b96"
server_key_iv: "08a8d872086feb175d7354c1f07a191b"
server_key_version: "encryptionkey@0"
I20260924 13:51:40.906392 2300 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790257629041645-13911-0/minicluster-data/master-0/wal/instance:
uuid: "9d1bd9ffa31948908abf5bd659b4bce6"
format_stamp: "Formatted at 2026-09-24 13:51:40 on dist-test-slave-sjl2"
server_key: "f1f0ee94a7e9ee2e824bbd6c74867b96"
server_key_iv: "08a8d872086feb175d7354c1f07a191b"
server_key_version: "encryptionkey@0"
I20260924 13:51:40.907735 2300 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260924 13:51:40.908447 2315 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 13:51:40.908618 2300 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260924 13:51:40.908700 2300 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790257629041645-13911-0/minicluster-data/master-0/data,/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790257629041645-13911-0/minicluster-data/master-0/wal
uuid: "9d1bd9ffa31948908abf5bd659b4bce6"
format_stamp: "Formatted at 2026-09-24 13:51:40 on dist-test-slave-sjl2"
server_key: "f1f0ee94a7e9ee2e824bbd6c74867b96"
server_key_iv: "08a8d872086feb175d7354c1f07a191b"
server_key_version: "encryptionkey@0"
I20260924 13:51:40.908763 2300 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790257629041645-13911-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790257629041645-13911-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790257629041645-13911-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 13:51:40.912015 2300 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 13:51:40.914096 2300 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 13:51:40.914247 2300 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 13:51:40.917960 2300 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35199
I20260924 13:51:40.917991 2367 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35199 every 8 connection(s)
I20260924 13:51:40.918304 2300 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790257629041645-13911-0/minicluster-data/master-0/data/info.pb
I20260924 13:51:40.919170 2368 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 13:51:40.921490 2368 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9d1bd9ffa31948908abf5bd659b4bce6: Bootstrap starting.
I20260924 13:51:40.922125 2368 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9d1bd9ffa31948908abf5bd659b4bce6: Neither blocks nor log segments found. Creating new log.
I20260924 13:51:40.922389 2368 log.cc:824] T 00000000000000000000000000000000 P 9d1bd9ffa31948908abf5bd659b4bce6: Log is configured to *not* fsync() on all Append() calls
I20260924 13:51:40.923081 2368 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9d1bd9ffa31948908abf5bd659b4bce6: No bootstrap required, opened a new log
I20260924 13:51:40.924687 2368 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9d1bd9ffa31948908abf5bd659b4bce6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d1bd9ffa31948908abf5bd659b4bce6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35199 } }
I20260924 13:51:40.924832 2368 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9d1bd9ffa31948908abf5bd659b4bce6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 13:51:40.924862 2368 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9d1bd9ffa31948908abf5bd659b4bce6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9d1bd9ffa31948908abf5bd659b4bce6, State: Initialized, Role: FOLLOWER
I20260924 13:51:40.925033 2368 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9d1bd9ffa31948908abf5bd659b4bce6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d1bd9ffa31948908abf5bd659b4bce6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35199 } }
I20260924 13:51:40.925097 2368 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9d1bd9ffa31948908abf5bd659b4bce6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 13:51:40.925119 2368 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9d1bd9ffa31948908abf5bd659b4bce6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 13:51:40.925164 2368 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9d1bd9ffa31948908abf5bd659b4bce6 [term 0 FOLLOWER]: Advancing to term 1
I20260924 13:51:40.925694 2368 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9d1bd9ffa31948908abf5bd659b4bce6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d1bd9ffa31948908abf5bd659b4bce6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35199 } }
I20260924 13:51:40.925786 2368 leader_election.cc:302] T 00000000000000000000000000000000 P 9d1bd9ffa31948908abf5bd659b4bce6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9d1bd9ffa31948908abf5bd659b4bce6; no voters:
I20260924 13:51:40.925938 2368 leader_election.cc:288] T 00000000000000000000000000000000 P 9d1bd9ffa31948908abf5bd659b4bce6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 13:51:40.925974 2371 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9d1bd9ffa31948908abf5bd659b4bce6 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 13:51:40.926117 2371 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9d1bd9ffa31948908abf5bd659b4bce6 [term 1 LEADER]: Becoming Leader. State: Replica: 9d1bd9ffa31948908abf5bd659b4bce6, State: Running, Role: LEADER
I20260924 13:51:40.926191 2368 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9d1bd9ffa31948908abf5bd659b4bce6 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 13:51:40.926205 2371 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9d1bd9ffa31948908abf5bd659b4bce6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d1bd9ffa31948908abf5bd659b4bce6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35199 } }
I20260924 13:51:40.926708 13911 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJN9WRF/build/release/bin/kudu as pid 2300
I20260924 13:51:40.926714 2372 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9d1bd9ffa31948908abf5bd659b4bce6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9d1bd9ffa31948908abf5bd659b4bce6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d1bd9ffa31948908abf5bd659b4bce6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35199 } } }
I20260924 13:51:40.926786 2372 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9d1bd9ffa31948908abf5bd659b4bce6 [sys.catalog]: This master's current role is: LEADER
I20260924 13:51:40.926834 13911 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790257629041645-13911-0/minicluster-data/master-0/wal/instance
I20260924 13:51:40.927047 13911 external_mini_cluster.cc:1428] Setting key dbdac4be8dc3c404a86197465eac51bc
I20260924 13:51:40.927278 2373 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9d1bd9ffa31948908abf5bd659b4bce6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9d1bd9ffa31948908abf5bd659b4bce6. Latest consensus state: current_term: 1 leader_uuid: "9d1bd9ffa31948908abf5bd659b4bce6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d1bd9ffa31948908abf5bd659b4bce6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35199 } } }
I20260924 13:51:40.927350 2373 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9d1bd9ffa31948908abf5bd659b4bce6 [sys.catalog]: This master's current role is: LEADER
I20260924 13:51:40.927433 2380 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 13:51:40.928009 2380 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 13:51:40.929226 2389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:40.927509 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45955 (local address 127.0.0.1:35199)
0924 13:51:40.927669 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0924 13:51:40.927679 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0924 13:51:40.927852 (+ 173us) server_negotiation.cc:407] Connection header received
0924 13:51:40.927991 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:51:40.927999 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:51:40.928033 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:51:40.928181 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:51:40.928756 (+ 575us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:51:40.928760 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:51:40.928760 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 13:51:40.928764 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:51:40.928801 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 13:51:40.928814 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:51:40.928867 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0924 13:51:40.928963 (+ 96us) server_negotiation.cc:299] Negotiation successful
0924 13:51:40.929008 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":102,"thread_start_us":73,"threads_started":1}
I20260924 13:51:40.929562 2380 catalog_manager.cc:1347] Generated new cluster ID: 247fef001f5f4a11823d03ef27752ba6
I20260924 13:51:40.929636 2380 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 13:51:40.930022 13911 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260924 13:51:40.932260 2389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:40.931344 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34763 (local address 127.0.0.1:35199)
0924 13:51:40.931378 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0924 13:51:40.931379 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0924 13:51:40.931386 (+ 7us) server_negotiation.cc:407] Connection header received
0924 13:51:40.931472 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:51:40.931473 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:51:40.931478 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:51:40.931527 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:51:40.932048 (+ 521us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:51:40.932050 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:51:40.932050 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 13:51:40.932054 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:51:40.932069 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 13:51:40.932074 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:51:40.932120 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0924 13:51:40.932170 (+ 50us) server_negotiation.cc:299] Negotiation successful
0924 13:51:40.932184 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260924 13:51:40.961490 2380 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 13:51:40.961680 2380 catalog_manager.cc:1504] Loading token signing keys...
I20260924 13:51:40.970114 2380 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9d1bd9ffa31948908abf5bd659b4bce6: Generated new TSK 0
I20260924 13:51:40.970305 2380 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260924 13:51:40.977136 13911 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJN9WRF/build/release/bin/kudu with pid 2300
2026-09-24T13:51:40Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1230 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790257629041645-13911-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 13:51:41 dist-test-slave-sjl2 krb5kdc[2399](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 13:51:41 dist-test-slave-sjl2 krb5kdc[2399](info): set up 2 sockets
Sep 24 13:51:41 dist-test-slave-sjl2 krb5kdc[2399](info): commencing operation
krb5kdc: starting...
W20260924 13:51:41.930075 13911 mini_kdc.cc:121] Time spent starting KDC: real 0.935s user 0.002s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790257629041645-13911-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790257629041645-13911-0/krb5kdc/test-user.keytab.
Sep 24 13:51:42 dist-test-slave-sjl2 krb5kdc[2399](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257902, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T13:51:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T13:51:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790257629041645-13911-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790257629041645-13911-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260924 13:51:42.063680 13911 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJN9WRF/build/release/bin/kudu
/tmp/dist-test-taskJN9WRF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790257629041645-13911-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790257629041645-13911-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790257629041645-13911-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790257629041645-13911-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45657
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790257629041645-13911-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38509
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45657
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790257629041645-13911-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790257629041645-13911-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790257629041645-13911-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 13:51:42.143157 2412 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 13:51:42.143361 2412 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 13:51:42.143383 2412 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 13:51:42.144496 2412 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 13:51:42.144536 2412 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 13:51:42.144548 2412 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 13:51:42.144559 2412 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 13:51:42.144574 2412 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 13:51:42.146063 2412 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38509
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790257629041645-13911-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790257629041645-13911-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45657
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790257629041645-13911-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45657
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790257629041645-13911-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2412
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790257629041645-13911-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 12:29:37 UTC on ccycloud.quasar-iqzbxn.root.comops.site
build id 4670106
I20260924 13:51:42.146260 2412 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 13:51:42.146428 2412 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 13:51:42.151094 2420 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 13:51:42.151094 2418 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 13:51:42.151146 2417 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 13:51:42.151121 2412 server_base.cc:1034] running on GCE node
I20260924 13:51:42.151559 2412 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 13:51:42.151865 2412 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 13:51:42.152979 2412 hybrid_clock.cc:648] HybridClock initialized: now 1790257902152958 us; error 37 us; skew 500 ppm
Sep 24 13:51:42 dist-test-slave-sjl2 krb5kdc[2399](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257902, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 13:51:42.155328 2412 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260924 13:51:42.155753 2412 webserver.cc:460] Webserver started at http://127.0.0.1:44679/ using document root <none> and password file <none>
I20260924 13:51:42.155962 2412 fs_manager.cc:359] Metadata directory not provided
I20260924 13:51:42.156010 2412 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 13:51:42.156134 2412 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 13:51:42.157222 2412 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790257629041645-13911-0/minicluster-data/master-0/data/instance:
uuid: "9a74cc80e6e647bd90b3973b22dda2af"
format_stamp: "Formatted at 2026-09-24 13:51:42 on dist-test-slave-sjl2"
server_key: "e983c9c39ec7c24dfdf122a89b99d0bb"
server_key_iv: "84c3049624057c7d652e29a85ed2d542"
server_key_version: "encryptionkey@0"
I20260924 13:51:42.157552 2412 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790257629041645-13911-0/minicluster-data/master-0/wal/instance:
uuid: "9a74cc80e6e647bd90b3973b22dda2af"
format_stamp: "Formatted at 2026-09-24 13:51:42 on dist-test-slave-sjl2"
server_key: "e983c9c39ec7c24dfdf122a89b99d0bb"
server_key_iv: "84c3049624057c7d652e29a85ed2d542"
server_key_version: "encryptionkey@0"
I20260924 13:51:42.158872 2412 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260924 13:51:42.159669 2427 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 13:51:42.159840 2412 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260924 13:51:42.159942 2412 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790257629041645-13911-0/minicluster-data/master-0/data,/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790257629041645-13911-0/minicluster-data/master-0/wal
uuid: "9a74cc80e6e647bd90b3973b22dda2af"
format_stamp: "Formatted at 2026-09-24 13:51:42 on dist-test-slave-sjl2"
server_key: "e983c9c39ec7c24dfdf122a89b99d0bb"
server_key_iv: "84c3049624057c7d652e29a85ed2d542"
server_key_version: "encryptionkey@0"
I20260924 13:51:42.160008 2412 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790257629041645-13911-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790257629041645-13911-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790257629041645-13911-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 13:51:42.170567 2412 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 13:51:42.172228 2412 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 13:51:42.172363 2412 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 13:51:42.176093 2412 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45657
I20260924 13:51:42.176152 2479 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45657 every 8 connection(s)
I20260924 13:51:42.176461 2412 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790257629041645-13911-0/minicluster-data/master-0/data/info.pb
I20260924 13:51:42.177525 2480 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 13:51:42.178268 13911 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJN9WRF/build/release/bin/kudu as pid 2412
I20260924 13:51:42.178380 13911 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790257629041645-13911-0/minicluster-data/master-0/wal/instance
I20260924 13:51:42.178551 13911 external_mini_cluster.cc:1428] Setting key c3a9e3e9b4ede867d7db0882b1b3fa91
I20260924 13:51:42.180243 2480 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9a74cc80e6e647bd90b3973b22dda2af: Bootstrap starting.
I20260924 13:51:42.180830 2480 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9a74cc80e6e647bd90b3973b22dda2af: Neither blocks nor log segments found. Creating new log.
I20260924 13:51:42.181154 2480 log.cc:824] T 00000000000000000000000000000000 P 9a74cc80e6e647bd90b3973b22dda2af: Log is configured to *not* fsync() on all Append() calls
I20260924 13:51:42.181845 2480 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9a74cc80e6e647bd90b3973b22dda2af: No bootstrap required, opened a new log
I20260924 13:51:42.183950 2480 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9a74cc80e6e647bd90b3973b22dda2af [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a74cc80e6e647bd90b3973b22dda2af" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45657 } }
I20260924 13:51:42.184129 2480 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9a74cc80e6e647bd90b3973b22dda2af [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 13:51:42.184156 2480 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9a74cc80e6e647bd90b3973b22dda2af [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9a74cc80e6e647bd90b3973b22dda2af, State: Initialized, Role: FOLLOWER
I20260924 13:51:42.184276 2480 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9a74cc80e6e647bd90b3973b22dda2af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a74cc80e6e647bd90b3973b22dda2af" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45657 } }
I20260924 13:51:42.184346 2480 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9a74cc80e6e647bd90b3973b22dda2af [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 13:51:42.184368 2480 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9a74cc80e6e647bd90b3973b22dda2af [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 13:51:42.184412 2480 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9a74cc80e6e647bd90b3973b22dda2af [term 0 FOLLOWER]: Advancing to term 1
I20260924 13:51:42.184942 2480 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9a74cc80e6e647bd90b3973b22dda2af [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a74cc80e6e647bd90b3973b22dda2af" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45657 } }
I20260924 13:51:42.185045 2480 leader_election.cc:302] T 00000000000000000000000000000000 P 9a74cc80e6e647bd90b3973b22dda2af [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9a74cc80e6e647bd90b3973b22dda2af; no voters:
I20260924 13:51:42.185202 2480 leader_election.cc:288] T 00000000000000000000000000000000 P 9a74cc80e6e647bd90b3973b22dda2af [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 13:51:42.185240 2485 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9a74cc80e6e647bd90b3973b22dda2af [term 1 FOLLOWER]: Leader election won for term 1
I20260924 13:51:42.185392 2485 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9a74cc80e6e647bd90b3973b22dda2af [term 1 LEADER]: Becoming Leader. State: Replica: 9a74cc80e6e647bd90b3973b22dda2af, State: Running, Role: LEADER
I20260924 13:51:42.185482 2485 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9a74cc80e6e647bd90b3973b22dda2af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a74cc80e6e647bd90b3973b22dda2af" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45657 } }
I20260924 13:51:42.185518 2480 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9a74cc80e6e647bd90b3973b22dda2af [sys.catalog]: configured and running, proceeding with master startup.
Sep 24 13:51:42 dist-test-slave-sjl2 krb5kdc[2399](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257902, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260924 13:51:42.185822 2485 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9a74cc80e6e647bd90b3973b22dda2af [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9a74cc80e6e647bd90b3973b22dda2af. Latest consensus state: current_term: 1 leader_uuid: "9a74cc80e6e647bd90b3973b22dda2af" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a74cc80e6e647bd90b3973b22dda2af" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45657 } } }
I20260924 13:51:42.185891 2485 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9a74cc80e6e647bd90b3973b22dda2af [sys.catalog]: This master's current role is: LEADER
I20260924 13:51:42.186594 2491 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 13:51:42.187071 2491 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 13:51:42.187268 2486 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9a74cc80e6e647bd90b3973b22dda2af [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9a74cc80e6e647bd90b3973b22dda2af" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a74cc80e6e647bd90b3973b22dda2af" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45657 } } }
I20260924 13:51:42.187980 2486 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9a74cc80e6e647bd90b3973b22dda2af [sys.catalog]: This master's current role is: LEADER
I20260924 13:51:42.188892 2491 catalog_manager.cc:1347] Generated new cluster ID: 474070bd3d8a4ba2965e8f84e3a0623a
I20260924 13:51:42.188972 2491 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 13:51:42.189975 2483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:42.179080 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.210:43567 (local address 127.0.0.1:45657)
0924 13:51:42.179264 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0924 13:51:42.179273 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0924 13:51:42.179305 (+ 32us) server_negotiation.cc:407] Connection header received
0924 13:51:42.179967 (+ 662us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:51:42.179976 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:51:42.180039 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:51:42.180182 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:51:42.180796 (+ 614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:51:42.182108 (+ 1312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:51:42.182852 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:51:42.183480 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:51:42.183596 (+ 116us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:51:42.186270 (+ 2674us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:51:42.186274 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:51:42.186274 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 13:51:42.186284 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:51:42.188297 (+ 2013us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:51:42.188779 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:51:42.188782 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:51:42.188783 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:51:42.188896 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:51:42.189151 (+ 255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:51:42.189152 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:51:42.189153 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:51:42.189387 (+ 234us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:51:42.189532 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0924 13:51:42.189676 (+ 144us) server_negotiation.cc:299] Negotiation successful
0924 13:51:42.189772 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135,"thread_start_us":86,"threads_started":1}
I20260924 13:51:42.190883 13911 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260924 13:51:42.197866 2483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:42.192139 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.210:53151 (local address 127.0.0.1:45657)
0924 13:51:42.192176 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0924 13:51:42.192178 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:51:42.192287 (+ 109us) server_negotiation.cc:407] Connection header received
0924 13:51:42.192338 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:51:42.192339 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:51:42.192348 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:51:42.192390 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:51:42.192856 (+ 466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:51:42.193399 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:51:42.194127 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:51:42.194359 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:51:42.194412 (+ 53us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:51:42.195561 (+ 1149us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:51:42.195562 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:51:42.195562 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 13:51:42.195570 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:51:42.196776 (+ 1206us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:51:42.197172 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:51:42.197173 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:51:42.197173 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:51:42.197222 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:51:42.197410 (+ 188us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:51:42.197410 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:51:42.197411 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:51:42.197510 (+ 99us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:51:42.197600 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0924 13:51:42.197715 (+ 115us) server_negotiation.cc:299] Negotiation successful
0924 13:51:42.197774 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260924 13:51:42.217161 2491 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 13:51:42.217732 2491 catalog_manager.cc:1504] Loading token signing keys...
I20260924 13:51:42.228088 2491 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9a74cc80e6e647bd90b3973b22dda2af: Generated new TSK 0
I20260924 13:51:42.228274 2491 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260924 13:51:42.235563 13911 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJN9WRF/build/release/bin/kudu with pid 2412
W20260924 13:51:42.239414 2397 connection.cc:535] client connection to 127.0.0.1:45657 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:45657): Connection reset by peer (error 104)
2026-09-24T13:51:42Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1258 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790257629041645-13911-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 13:51:42 dist-test-slave-sjl2 krb5kdc[2511](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 13:51:42 dist-test-slave-sjl2 krb5kdc[2511](info): set up 2 sockets
Sep 24 13:51:42 dist-test-slave-sjl2 krb5kdc[2511](info): commencing operation
krb5kdc: starting...
W20260924 13:51:43.177934 13911 mini_kdc.cc:121] Time spent starting KDC: real 0.925s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790257629041645-13911-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790257629041645-13911-0/krb5kdc/test-user.keytab.
Sep 24 13:51:43 dist-test-slave-sjl2 krb5kdc[2511](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257903, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T13:51:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T13:51:43Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790257629041645-13911-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790257629041645-13911-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260924 13:51:43.295902 13911 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJN9WRF/build/release/bin/kudu
/tmp/dist-test-taskJN9WRF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790257629041645-13911-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790257629041645-13911-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790257629041645-13911-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790257629041645-13911-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39217
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790257629041645-13911-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35507
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39217
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790257629041645-13911-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790257629041645-13911-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790257629041645-13911-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 13:51:43.375586 2524 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 13:51:43.375813 2524 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 13:51:43.375837 2524 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 13:51:43.377102 2524 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 13:51:43.377153 2524 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 13:51:43.377167 2524 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 13:51:43.377178 2524 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 13:51:43.377189 2524 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 13:51:43.378744 2524 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35507
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790257629041645-13911-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790257629041645-13911-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39217
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790257629041645-13911-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39217
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790257629041645-13911-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2524
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790257629041645-13911-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 12:29:37 UTC on ccycloud.quasar-iqzbxn.root.comops.site
build id 4670106
I20260924 13:51:43.378988 2524 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 13:51:43.379206 2524 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 13:51:43.383915 2532 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 13:51:43.383991 2529 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 13:51:43.383934 2524 server_base.cc:1034] running on GCE node
W20260924 13:51:43.384092 2530 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 13:51:43.384491 2524 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 13:51:43.384775 2524 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 13:51:43.385928 2524 hybrid_clock.cc:648] HybridClock initialized: now 1790257903385911 us; error 31 us; skew 500 ppm
Sep 24 13:51:43 dist-test-slave-sjl2 krb5kdc[2511](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257903, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 13:51:43.388316 2524 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260924 13:51:43.388762 2524 webserver.cc:460] Webserver started at http://127.0.0.1:33485/ using document root <none> and password file <none>
I20260924 13:51:43.389017 2524 fs_manager.cc:359] Metadata directory not provided
I20260924 13:51:43.389067 2524 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 13:51:43.389187 2524 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 13:51:43.390277 2524 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790257629041645-13911-0/minicluster-data/master-0/data/instance:
uuid: "4a0abdfc77b84a6096691d190b1f088c"
format_stamp: "Formatted at 2026-09-24 13:51:43 on dist-test-slave-sjl2"
server_key: "fbae108d3b6eedd6d54a7c3c25b69f40"
server_key_iv: "19f72d2468497b064592d7cd6ec0bf06"
server_key_version: "encryptionkey@0"
I20260924 13:51:43.390654 2524 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790257629041645-13911-0/minicluster-data/master-0/wal/instance:
uuid: "4a0abdfc77b84a6096691d190b1f088c"
format_stamp: "Formatted at 2026-09-24 13:51:43 on dist-test-slave-sjl2"
server_key: "fbae108d3b6eedd6d54a7c3c25b69f40"
server_key_iv: "19f72d2468497b064592d7cd6ec0bf06"
server_key_version: "encryptionkey@0"
I20260924 13:51:43.392002 2524 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260924 13:51:43.392756 2539 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 13:51:43.392980 2524 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260924 13:51:43.393076 2524 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790257629041645-13911-0/minicluster-data/master-0/data,/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790257629041645-13911-0/minicluster-data/master-0/wal
uuid: "4a0abdfc77b84a6096691d190b1f088c"
format_stamp: "Formatted at 2026-09-24 13:51:43 on dist-test-slave-sjl2"
server_key: "fbae108d3b6eedd6d54a7c3c25b69f40"
server_key_iv: "19f72d2468497b064592d7cd6ec0bf06"
server_key_version: "encryptionkey@0"
I20260924 13:51:43.393149 2524 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790257629041645-13911-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790257629041645-13911-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790257629041645-13911-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 13:51:43.415043 2524 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 13:51:43.417053 2524 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 13:51:43.417188 2524 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 13:51:43.420869 2524 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39217
I20260924 13:51:43.420991 2591 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39217 every 8 connection(s)
I20260924 13:51:43.421278 2524 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790257629041645-13911-0/minicluster-data/master-0/data/info.pb
I20260924 13:51:43.422219 2592 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 13:51:43.424644 2592 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4a0abdfc77b84a6096691d190b1f088c: Bootstrap starting.
I20260924 13:51:43.425263 2592 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4a0abdfc77b84a6096691d190b1f088c: Neither blocks nor log segments found. Creating new log.
I20260924 13:51:43.425499 2592 log.cc:824] T 00000000000000000000000000000000 P 4a0abdfc77b84a6096691d190b1f088c: Log is configured to *not* fsync() on all Append() calls
I20260924 13:51:43.426174 2592 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4a0abdfc77b84a6096691d190b1f088c: No bootstrap required, opened a new log
I20260924 13:51:43.427714 2592 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4a0abdfc77b84a6096691d190b1f088c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a0abdfc77b84a6096691d190b1f088c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39217 } }
I20260924 13:51:43.427856 2592 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4a0abdfc77b84a6096691d190b1f088c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 13:51:43.427881 2592 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4a0abdfc77b84a6096691d190b1f088c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4a0abdfc77b84a6096691d190b1f088c, State: Initialized, Role: FOLLOWER
I20260924 13:51:43.427989 2592 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4a0abdfc77b84a6096691d190b1f088c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a0abdfc77b84a6096691d190b1f088c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39217 } }
I20260924 13:51:43.428043 2592 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4a0abdfc77b84a6096691d190b1f088c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 13:51:43.428079 2592 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4a0abdfc77b84a6096691d190b1f088c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 13:51:43.428115 2592 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4a0abdfc77b84a6096691d190b1f088c [term 0 FOLLOWER]: Advancing to term 1
I20260924 13:51:43.428613 2592 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4a0abdfc77b84a6096691d190b1f088c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a0abdfc77b84a6096691d190b1f088c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39217 } }
I20260924 13:51:43.428700 2592 leader_election.cc:302] T 00000000000000000000000000000000 P 4a0abdfc77b84a6096691d190b1f088c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4a0abdfc77b84a6096691d190b1f088c; no voters:
I20260924 13:51:43.428825 2592 leader_election.cc:288] T 00000000000000000000000000000000 P 4a0abdfc77b84a6096691d190b1f088c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 13:51:43.428857 2595 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4a0abdfc77b84a6096691d190b1f088c [term 1 FOLLOWER]: Leader election won for term 1
I20260924 13:51:43.429028 2595 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4a0abdfc77b84a6096691d190b1f088c [term 1 LEADER]: Becoming Leader. State: Replica: 4a0abdfc77b84a6096691d190b1f088c, State: Running, Role: LEADER
I20260924 13:51:43.429095 2592 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4a0abdfc77b84a6096691d190b1f088c [sys.catalog]: configured and running, proceeding with master startup.
I20260924 13:51:43.429121 2595 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4a0abdfc77b84a6096691d190b1f088c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a0abdfc77b84a6096691d190b1f088c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39217 } }
I20260924 13:51:43.429462 2597 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4a0abdfc77b84a6096691d190b1f088c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4a0abdfc77b84a6096691d190b1f088c. Latest consensus state: current_term: 1 leader_uuid: "4a0abdfc77b84a6096691d190b1f088c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a0abdfc77b84a6096691d190b1f088c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39217 } } }
I20260924 13:51:43.429543 2597 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4a0abdfc77b84a6096691d190b1f088c [sys.catalog]: This master's current role is: LEADER
I20260924 13:51:43.429616 2596 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4a0abdfc77b84a6096691d190b1f088c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4a0abdfc77b84a6096691d190b1f088c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a0abdfc77b84a6096691d190b1f088c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39217 } } }
I20260924 13:51:43.429677 2596 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4a0abdfc77b84a6096691d190b1f088c [sys.catalog]: This master's current role is: LEADER
I20260924 13:51:43.429738 2602 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 13:51:43.430294 2602 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 13:51:43.430927 13911 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJN9WRF/build/release/bin/kudu as pid 2524
I20260924 13:51:43.431056 13911 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790257629041645-13911-0/minicluster-data/master-0/wal/instance
I20260924 13:51:43.431347 13911 external_mini_cluster.cc:1428] Setting key d1843aa71144c7fcff6056160f9cb56a
I20260924 13:51:43.432240 2602 catalog_manager.cc:1347] Generated new cluster ID: 5cfea58afbca4e128eb13314301b8a6a
I20260924 13:51:43.432286 2602 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 13:51:43.438122 2615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:43.431840 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.210:48199 (local address 127.0.0.1:39217)
0924 13:51:43.432320 (+ 480us) server_negotiation.cc:206] Beginning negotiation
0924 13:51:43.432329 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0924 13:51:43.432359 (+ 30us) server_negotiation.cc:407] Connection header received
0924 13:51:43.432467 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:51:43.432473 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:51:43.432529 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:51:43.432663 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:51:43.433029 (+ 366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:51:43.434448 (+ 1419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:51:43.435571 (+ 1123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:51:43.436212 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:51:43.436321 (+ 109us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:51:43.437273 (+ 952us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:51:43.437277 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:51:43.437278 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 13:51:43.437490 (+ 212us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:51:43.437534 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 13:51:43.437550 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:51:43.437625 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0924 13:51:43.437889 (+ 264us) server_negotiation.cc:299] Negotiation successful
0924 13:51:43.437918 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":441,"thread_start_us":75,"threads_started":1}
I20260924 13:51:43.438997 13911 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260924 13:51:43.440990 2602 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 13:51:43.441457 2602 catalog_manager.cc:1504] Loading token signing keys...
I20260924 13:51:43.443945 2602 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4a0abdfc77b84a6096691d190b1f088c: Generated new TSK 0
I20260924 13:51:43.444108 2602 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260924 13:51:43.444268 2615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:43.440583 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.210:36087 (local address 127.0.0.1:39217)
0924 13:51:43.440651 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0924 13:51:43.440653 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:51:43.440665 (+ 12us) server_negotiation.cc:407] Connection header received
0924 13:51:43.440687 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:51:43.440688 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:51:43.440700 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:51:43.440779 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:51:43.440955 (+ 176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:51:43.441692 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:51:43.442731 (+ 1039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:51:43.443050 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:51:43.443087 (+ 37us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:51:43.443870 (+ 783us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:51:43.443871 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:51:43.443871 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 13:51:43.443875 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:51:43.443890 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 13:51:43.443895 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:51:43.443952 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0924 13:51:43.444093 (+ 141us) server_negotiation.cc:299] Negotiation successful
0924 13:51:43.444108 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260924 13:51:43.446376 13911 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJN9WRF/build/release/bin/kudu with pid 2524
W20260924 13:51:43.450261 2509 connection.cc:535] client connection to 127.0.0.1:39217 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:39217): Connection reset by peer (error 104)
2026-09-24T13:51:43Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1210 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790257629041645-13911-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 13:51:43 dist-test-slave-sjl2 krb5kdc[2623](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 13:51:43 dist-test-slave-sjl2 krb5kdc[2623](info): set up 2 sockets
Sep 24 13:51:43 dist-test-slave-sjl2 krb5kdc[2623](info): commencing operation
krb5kdc: starting...
W20260924 13:51:44.402036 13911 mini_kdc.cc:121] Time spent starting KDC: real 0.938s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790257629041645-13911-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790257629041645-13911-0/krb5kdc/test-user.keytab.
Sep 24 13:51:44 dist-test-slave-sjl2 krb5kdc[2623](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257904, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T13:51:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T13:51:44Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790257629041645-13911-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790257629041645-13911-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260924 13:51:44.517808 13911 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJN9WRF/build/release/bin/kudu
/tmp/dist-test-taskJN9WRF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790257629041645-13911-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790257629041645-13911-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790257629041645-13911-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790257629041645-13911-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34691
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790257629041645-13911-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39947
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34691
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790257629041645-13911-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790257629041645-13911-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790257629041645-13911-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 13:51:44.596321 2636 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 13:51:44.596540 2636 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 13:51:44.596573 2636 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 13:51:44.597800 2636 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 13:51:44.597849 2636 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 13:51:44.597868 2636 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 13:51:44.597884 2636 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 13:51:44.597899 2636 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 13:51:44.599404 2636 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39947
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790257629041645-13911-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790257629041645-13911-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34691
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790257629041645-13911-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34691
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790257629041645-13911-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2636
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790257629041645-13911-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 12:29:37 UTC on ccycloud.quasar-iqzbxn.root.comops.site
build id 4670106
I20260924 13:51:44.599660 2636 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 13:51:44.599874 2636 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 13:51:44.604892 2641 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 13:51:44.604887 2644 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 13:51:44.604887 2642 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 13:51:44.604907 2636 server_base.cc:1034] running on GCE node
I20260924 13:51:44.605440 2636 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 13:51:44.605774 2636 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 13:51:44.606912 2636 hybrid_clock.cc:648] HybridClock initialized: now 1790257904606905 us; error 29 us; skew 500 ppm
Sep 24 13:51:44 dist-test-slave-sjl2 krb5kdc[2623](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257904, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 13:51:44.609273 2636 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260924 13:51:44.609665 2636 webserver.cc:460] Webserver started at http://127.0.0.1:36279/ using document root <none> and password file <none>
I20260924 13:51:44.609855 2636 fs_manager.cc:359] Metadata directory not provided
I20260924 13:51:44.609895 2636 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 13:51:44.609979 2636 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 13:51:44.610965 2636 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790257629041645-13911-0/minicluster-data/master-0/data/instance:
uuid: "016ba51eb3fe45699f522b50c34ad9d9"
format_stamp: "Formatted at 2026-09-24 13:51:44 on dist-test-slave-sjl2"
server_key: "3b8a82469a50a3684deb5793067b775d"
server_key_iv: "dbc2a895ac387d1c9e6175fb8061420f"
server_key_version: "encryptionkey@0"
I20260924 13:51:44.611241 2636 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790257629041645-13911-0/minicluster-data/master-0/wal/instance:
uuid: "016ba51eb3fe45699f522b50c34ad9d9"
format_stamp: "Formatted at 2026-09-24 13:51:44 on dist-test-slave-sjl2"
server_key: "3b8a82469a50a3684deb5793067b775d"
server_key_iv: "dbc2a895ac387d1c9e6175fb8061420f"
server_key_version: "encryptionkey@0"
I20260924 13:51:44.612470 2636 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260924 13:51:44.613132 2651 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 13:51:44.613278 2636 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 13:51:44.613359 2636 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790257629041645-13911-0/minicluster-data/master-0/data,/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790257629041645-13911-0/minicluster-data/master-0/wal
uuid: "016ba51eb3fe45699f522b50c34ad9d9"
format_stamp: "Formatted at 2026-09-24 13:51:44 on dist-test-slave-sjl2"
server_key: "3b8a82469a50a3684deb5793067b775d"
server_key_iv: "dbc2a895ac387d1c9e6175fb8061420f"
server_key_version: "encryptionkey@0"
I20260924 13:51:44.613420 2636 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790257629041645-13911-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790257629041645-13911-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790257629041645-13911-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 13:51:44.616469 2636 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 13:51:44.618480 2636 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 13:51:44.618607 2636 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 13:51:44.622274 2636 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34691
I20260924 13:51:44.622375 2703 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34691 every 8 connection(s)
I20260924 13:51:44.622627 2636 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790257629041645-13911-0/minicluster-data/master-0/data/info.pb
I20260924 13:51:44.623528 2704 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 13:51:44.625901 2704 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 016ba51eb3fe45699f522b50c34ad9d9: Bootstrap starting.
I20260924 13:51:44.626431 2704 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 016ba51eb3fe45699f522b50c34ad9d9: Neither blocks nor log segments found. Creating new log.
I20260924 13:51:44.626655 2704 log.cc:824] T 00000000000000000000000000000000 P 016ba51eb3fe45699f522b50c34ad9d9: Log is configured to *not* fsync() on all Append() calls
I20260924 13:51:44.627393 2704 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 016ba51eb3fe45699f522b50c34ad9d9: No bootstrap required, opened a new log
I20260924 13:51:44.629029 2704 raft_consensus.cc:348] T 00000000000000000000000000000000 P 016ba51eb3fe45699f522b50c34ad9d9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "016ba51eb3fe45699f522b50c34ad9d9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34691 } }
I20260924 13:51:44.629194 2704 raft_consensus.cc:374] T 00000000000000000000000000000000 P 016ba51eb3fe45699f522b50c34ad9d9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 13:51:44.629217 2704 raft_consensus.cc:729] T 00000000000000000000000000000000 P 016ba51eb3fe45699f522b50c34ad9d9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 016ba51eb3fe45699f522b50c34ad9d9, State: Initialized, Role: FOLLOWER
I20260924 13:51:44.629334 2704 consensus_queue.cc:260] T 00000000000000000000000000000000 P 016ba51eb3fe45699f522b50c34ad9d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "016ba51eb3fe45699f522b50c34ad9d9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34691 } }
I20260924 13:51:44.629386 2704 raft_consensus.cc:388] T 00000000000000000000000000000000 P 016ba51eb3fe45699f522b50c34ad9d9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 13:51:44.629405 2704 raft_consensus.cc:482] T 00000000000000000000000000000000 P 016ba51eb3fe45699f522b50c34ad9d9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 13:51:44.629437 2704 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 016ba51eb3fe45699f522b50c34ad9d9 [term 0 FOLLOWER]: Advancing to term 1
I20260924 13:51:44.629956 2704 raft_consensus.cc:504] T 00000000000000000000000000000000 P 016ba51eb3fe45699f522b50c34ad9d9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "016ba51eb3fe45699f522b50c34ad9d9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34691 } }
I20260924 13:51:44.630052 2704 leader_election.cc:302] T 00000000000000000000000000000000 P 016ba51eb3fe45699f522b50c34ad9d9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 016ba51eb3fe45699f522b50c34ad9d9; no voters:
I20260924 13:51:44.630201 2704 leader_election.cc:288] T 00000000000000000000000000000000 P 016ba51eb3fe45699f522b50c34ad9d9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 13:51:44.630264 2707 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 016ba51eb3fe45699f522b50c34ad9d9 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 13:51:44.630426 2707 raft_consensus.cc:686] T 00000000000000000000000000000000 P 016ba51eb3fe45699f522b50c34ad9d9 [term 1 LEADER]: Becoming Leader. State: Replica: 016ba51eb3fe45699f522b50c34ad9d9, State: Running, Role: LEADER
I20260924 13:51:44.630434 2704 sys_catalog.cc:564] T 00000000000000000000000000000000 P 016ba51eb3fe45699f522b50c34ad9d9 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 13:51:44.630527 2707 consensus_queue.cc:237] T 00000000000000000000000000000000 P 016ba51eb3fe45699f522b50c34ad9d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "016ba51eb3fe45699f522b50c34ad9d9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34691 } }
I20260924 13:51:44.631047 2708 sys_catalog.cc:455] T 00000000000000000000000000000000 P 016ba51eb3fe45699f522b50c34ad9d9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "016ba51eb3fe45699f522b50c34ad9d9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "016ba51eb3fe45699f522b50c34ad9d9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34691 } } }
I20260924 13:51:44.631131 2708 sys_catalog.cc:458] T 00000000000000000000000000000000 P 016ba51eb3fe45699f522b50c34ad9d9 [sys.catalog]: This master's current role is: LEADER
I20260924 13:51:44.631292 2709 sys_catalog.cc:455] T 00000000000000000000000000000000 P 016ba51eb3fe45699f522b50c34ad9d9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 016ba51eb3fe45699f522b50c34ad9d9. Latest consensus state: current_term: 1 leader_uuid: "016ba51eb3fe45699f522b50c34ad9d9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "016ba51eb3fe45699f522b50c34ad9d9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34691 } } }
I20260924 13:51:44.631331 2716 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 13:51:44.631392 2709 sys_catalog.cc:458] T 00000000000000000000000000000000 P 016ba51eb3fe45699f522b50c34ad9d9 [sys.catalog]: This master's current role is: LEADER
I20260924 13:51:44.631911 2716 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 13:51:44.632390 13911 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJN9WRF/build/release/bin/kudu as pid 2636
I20260924 13:51:44.632486 13911 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790257629041645-13911-0/minicluster-data/master-0/wal/instance
I20260924 13:51:44.632666 13911 external_mini_cluster.cc:1428] Setting key 11a0a86cb07a894267c17db92c515d77
I20260924 13:51:44.633671 2716 catalog_manager.cc:1347] Generated new cluster ID: 4694ed4b29414879bd2fb7c2ad912e42
I20260924 13:51:44.633736 2716 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 13:51:44.635138 2726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:44.633111 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.210:48079 (local address 127.0.0.1:34691)
0924 13:51:44.633383 (+ 272us) server_negotiation.cc:206] Beginning negotiation
0924 13:51:44.633393 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0924 13:51:44.633426 (+ 33us) server_negotiation.cc:407] Connection header received
0924 13:51:44.633540 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:51:44.633548 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:51:44.633578 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:51:44.633713 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:51:44.634713 (+ 1000us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:51:44.634717 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:51:44.634717 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 13:51:44.634722 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:51:44.634765 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 13:51:44.634791 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:51:44.634864 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0924 13:51:44.634918 (+ 54us) server_negotiation.cc:299] Negotiation successful
0924 13:51:44.634963 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":227,"thread_start_us":63,"threads_started":1}
I20260924 13:51:44.636018 13911 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260924 13:51:44.638777 2726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:44.637539 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.210:49503 (local address 127.0.0.1:34691)
0924 13:51:44.637575 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0924 13:51:44.637577 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:51:44.637742 (+ 165us) server_negotiation.cc:407] Connection header received
0924 13:51:44.637845 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:51:44.637847 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:51:44.637860 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:51:44.637888 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:51:44.638544 (+ 656us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:51:44.638545 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:51:44.638545 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 13:51:44.638550 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:51:44.638565 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 13:51:44.638569 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:51:44.638599 (+ 30us) server_negotiation.cc:1035] Waiting for connection context
0924 13:51:44.638699 (+ 100us) server_negotiation.cc:299] Negotiation successful
0924 13:51:44.638713 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260924 13:51:44.658036 2716 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 13:51:44.658222 2716 catalog_manager.cc:1504] Loading token signing keys...
I20260924 13:51:44.666468 2716 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 016ba51eb3fe45699f522b50c34ad9d9: Generated new TSK 0
I20260924 13:51:44.666648 2716 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260924 13:51:44.673909 13911 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJN9WRF/build/release/bin/kudu with pid 2636
2026-09-24T13:51:44Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1228 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790257629041645-13911-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 13:51:44 dist-test-slave-sjl2 krb5kdc[2735](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 13:51:44 dist-test-slave-sjl2 krb5kdc[2735](info): set up 2 sockets
Sep 24 13:51:44 dist-test-slave-sjl2 krb5kdc[2735](info): commencing operation
krb5kdc: starting...
W20260924 13:51:45.621997 13911 mini_kdc.cc:121] Time spent starting KDC: real 0.930s user 0.001s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790257629041645-13911-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790257629041645-13911-0/krb5kdc/test-user.keytab.
Sep 24 13:51:45 dist-test-slave-sjl2 krb5kdc[2735](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257905, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T13:51:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T13:51:45Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790257629041645-13911-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790257629041645-13911-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260924 13:51:45.737815 13911 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJN9WRF/build/release/bin/kudu
/tmp/dist-test-taskJN9WRF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790257629041645-13911-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790257629041645-13911-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790257629041645-13911-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790257629041645-13911-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38919
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790257629041645-13911-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:46397
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38919
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790257629041645-13911-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790257629041645-13911-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790257629041645-13911-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 13:51:45.818071 2748 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 13:51:45.818282 2748 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 13:51:45.818315 2748 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 13:51:45.819474 2748 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 13:51:45.819515 2748 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 13:51:45.819535 2748 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 13:51:45.819550 2748 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 13:51:45.819566 2748 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 13:51:45.821039 2748 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:46397
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790257629041645-13911-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790257629041645-13911-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38919
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790257629041645-13911-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38919
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790257629041645-13911-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2748
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790257629041645-13911-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 12:29:37 UTC on ccycloud.quasar-iqzbxn.root.comops.site
build id 4670106
I20260924 13:51:45.821280 2748 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 13:51:45.821501 2748 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260924 13:51:45.826221 2748 server_base.cc:1034] running on GCE node
W20260924 13:51:45.826174 2756 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 13:51:45.826210 2754 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 13:51:45.826174 2753 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 13:51:45.826606 2748 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 13:51:45.826889 2748 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 13:51:45.828043 2748 hybrid_clock.cc:648] HybridClock initialized: now 1790257905828032 us; error 30 us; skew 500 ppm
Sep 24 13:51:45 dist-test-slave-sjl2 krb5kdc[2735](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257905, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 13:51:45.830519 2748 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260924 13:51:45.830943 2748 webserver.cc:460] Webserver started at http://127.0.0.1:42557/ using document root <none> and password file <none>
I20260924 13:51:45.831168 2748 fs_manager.cc:359] Metadata directory not provided
I20260924 13:51:45.831216 2748 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 13:51:45.831341 2748 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 13:51:45.832489 2748 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790257629041645-13911-0/minicluster-data/master-0/data/instance:
uuid: "a4e07ee1cf9b4d69a506cc049b141c54"
format_stamp: "Formatted at 2026-09-24 13:51:45 on dist-test-slave-sjl2"
server_key: "994eebd0201110ee2c3fe5a599c5f105"
server_key_iv: "94bd2eb8b823f2917d74f418b5799f12"
server_key_version: "encryptionkey@0"
I20260924 13:51:45.832813 2748 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790257629041645-13911-0/minicluster-data/master-0/wal/instance:
uuid: "a4e07ee1cf9b4d69a506cc049b141c54"
format_stamp: "Formatted at 2026-09-24 13:51:45 on dist-test-slave-sjl2"
server_key: "994eebd0201110ee2c3fe5a599c5f105"
server_key_iv: "94bd2eb8b823f2917d74f418b5799f12"
server_key_version: "encryptionkey@0"
I20260924 13:51:45.834077 2748 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260924 13:51:45.834753 2763 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 13:51:45.834887 2748 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260924 13:51:45.834964 2748 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790257629041645-13911-0/minicluster-data/master-0/data,/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790257629041645-13911-0/minicluster-data/master-0/wal
uuid: "a4e07ee1cf9b4d69a506cc049b141c54"
format_stamp: "Formatted at 2026-09-24 13:51:45 on dist-test-slave-sjl2"
server_key: "994eebd0201110ee2c3fe5a599c5f105"
server_key_iv: "94bd2eb8b823f2917d74f418b5799f12"
server_key_version: "encryptionkey@0"
I20260924 13:51:45.835023 2748 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790257629041645-13911-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790257629041645-13911-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790257629041645-13911-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 13:51:45.867295 2748 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 13:51:45.869424 2748 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 13:51:45.869565 2748 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 13:51:45.873543 2748 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38919
I20260924 13:51:45.873586 2815 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38919 every 8 connection(s)
I20260924 13:51:45.873906 2748 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790257629041645-13911-0/minicluster-data/master-0/data/info.pb
I20260924 13:51:45.874814 2816 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 13:51:45.877154 2816 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a4e07ee1cf9b4d69a506cc049b141c54: Bootstrap starting.
I20260924 13:51:45.877739 2816 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a4e07ee1cf9b4d69a506cc049b141c54: Neither blocks nor log segments found. Creating new log.
I20260924 13:51:45.878017 2816 log.cc:824] T 00000000000000000000000000000000 P a4e07ee1cf9b4d69a506cc049b141c54: Log is configured to *not* fsync() on all Append() calls
I20260924 13:51:45.878743 2816 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a4e07ee1cf9b4d69a506cc049b141c54: No bootstrap required, opened a new log
I20260924 13:51:45.880370 2816 raft_consensus.cc:348] T 00000000000000000000000000000000 P a4e07ee1cf9b4d69a506cc049b141c54 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4e07ee1cf9b4d69a506cc049b141c54" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38919 } }
I20260924 13:51:45.880532 2816 raft_consensus.cc:374] T 00000000000000000000000000000000 P a4e07ee1cf9b4d69a506cc049b141c54 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 13:51:45.880555 2816 raft_consensus.cc:729] T 00000000000000000000000000000000 P a4e07ee1cf9b4d69a506cc049b141c54 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a4e07ee1cf9b4d69a506cc049b141c54, State: Initialized, Role: FOLLOWER
I20260924 13:51:45.880695 2816 consensus_queue.cc:260] T 00000000000000000000000000000000 P a4e07ee1cf9b4d69a506cc049b141c54 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4e07ee1cf9b4d69a506cc049b141c54" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38919 } }
I20260924 13:51:45.880770 2816 raft_consensus.cc:388] T 00000000000000000000000000000000 P a4e07ee1cf9b4d69a506cc049b141c54 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 13:51:45.880791 2816 raft_consensus.cc:482] T 00000000000000000000000000000000 P a4e07ee1cf9b4d69a506cc049b141c54 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 13:51:45.880826 2816 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a4e07ee1cf9b4d69a506cc049b141c54 [term 0 FOLLOWER]: Advancing to term 1
I20260924 13:51:45.881424 2816 raft_consensus.cc:504] T 00000000000000000000000000000000 P a4e07ee1cf9b4d69a506cc049b141c54 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4e07ee1cf9b4d69a506cc049b141c54" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38919 } }
I20260924 13:51:45.881520 2816 leader_election.cc:302] T 00000000000000000000000000000000 P a4e07ee1cf9b4d69a506cc049b141c54 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a4e07ee1cf9b4d69a506cc049b141c54; no voters:
I20260924 13:51:45.881697 2816 leader_election.cc:288] T 00000000000000000000000000000000 P a4e07ee1cf9b4d69a506cc049b141c54 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 13:51:45.881739 2819 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a4e07ee1cf9b4d69a506cc049b141c54 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 13:51:45.881933 2819 raft_consensus.cc:686] T 00000000000000000000000000000000 P a4e07ee1cf9b4d69a506cc049b141c54 [term 1 LEADER]: Becoming Leader. State: Replica: a4e07ee1cf9b4d69a506cc049b141c54, State: Running, Role: LEADER
I20260924 13:51:45.881982 2816 sys_catalog.cc:564] T 00000000000000000000000000000000 P a4e07ee1cf9b4d69a506cc049b141c54 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 13:51:45.882037 2819 consensus_queue.cc:237] T 00000000000000000000000000000000 P a4e07ee1cf9b4d69a506cc049b141c54 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4e07ee1cf9b4d69a506cc049b141c54" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38919 } }
I20260924 13:51:45.882599 2821 sys_catalog.cc:455] T 00000000000000000000000000000000 P a4e07ee1cf9b4d69a506cc049b141c54 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a4e07ee1cf9b4d69a506cc049b141c54. Latest consensus state: current_term: 1 leader_uuid: "a4e07ee1cf9b4d69a506cc049b141c54" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4e07ee1cf9b4d69a506cc049b141c54" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38919 } } }
I20260924 13:51:45.882692 2821 sys_catalog.cc:458] T 00000000000000000000000000000000 P a4e07ee1cf9b4d69a506cc049b141c54 [sys.catalog]: This master's current role is: LEADER
I20260924 13:51:45.882753 13911 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJN9WRF/build/release/bin/kudu as pid 2748
I20260924 13:51:45.882741 2820 sys_catalog.cc:455] T 00000000000000000000000000000000 P a4e07ee1cf9b4d69a506cc049b141c54 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a4e07ee1cf9b4d69a506cc049b141c54" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4e07ee1cf9b4d69a506cc049b141c54" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38919 } } }
I20260924 13:51:45.882836 2820 sys_catalog.cc:458] T 00000000000000000000000000000000 P a4e07ee1cf9b4d69a506cc049b141c54 [sys.catalog]: This master's current role is: LEADER
I20260924 13:51:45.882872 13911 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790257629041645-13911-0/minicluster-data/master-0/wal/instance
I20260924 13:51:45.883023 2828 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 13:51:45.883092 13911 external_mini_cluster.cc:1428] Setting key b364c1fa0a3b3ac40615cf8fb3efdb2f
I20260924 13:51:45.883628 2828 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 13:51:45.885583 2828 catalog_manager.cc:1347] Generated new cluster ID: a08e6ac39be5442db7f969afb0f8be37
I20260924 13:51:45.885638 2828 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 24 13:51:45 dist-test-slave-sjl2 krb5kdc[2735](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257905, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260924 13:51:45.895920 2838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:45.883730 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.210:51287 (local address 127.0.0.1:38919)
0924 13:51:45.883941 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0924 13:51:45.883950 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0924 13:51:45.884147 (+ 197us) server_negotiation.cc:407] Connection header received
0924 13:51:45.884460 (+ 313us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:51:45.884467 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:51:45.884531 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:51:45.884679 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:51:45.885573 (+ 894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:51:45.886992 (+ 1419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:51:45.888083 (+ 1091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:51:45.888748 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:51:45.888865 (+ 117us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:51:45.891860 (+ 2995us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:51:45.891866 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:51:45.891866 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 13:51:45.891881 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:51:45.893947 (+ 2066us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:51:45.894474 (+ 527us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:51:45.894478 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:51:45.894478 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:51:45.894604 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:51:45.894826 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:51:45.894827 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:51:45.894827 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:51:45.895137 (+ 310us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:51:45.895289 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0924 13:51:45.895420 (+ 131us) server_negotiation.cc:299] Negotiation successful
0924 13:51:45.895663 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":162,"thread_start_us":106,"threads_started":1}
I20260924 13:51:45.897213 13911 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260924 13:51:45.901501 2828 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 13:51:45.902230 2828 catalog_manager.cc:1504] Loading token signing keys...
I20260924 13:51:45.909184 2838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:45.901246 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.210:43607 (local address 127.0.0.1:38919)
0924 13:51:45.901307 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0924 13:51:45.901309 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:51:45.902425 (+ 1116us) server_negotiation.cc:407] Connection header received
0924 13:51:45.902457 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:51:45.902458 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:51:45.902470 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:51:45.902531 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:51:45.903150 (+ 619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:51:45.903982 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:51:45.904766 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:51:45.905064 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:51:45.905132 (+ 68us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:51:45.906436 (+ 1304us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:51:45.906437 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:51:45.906437 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 13:51:45.906446 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:51:45.907834 (+ 1388us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:51:45.908249 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:51:45.908250 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:51:45.908251 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:51:45.908311 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:51:45.908585 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:51:45.908586 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:51:45.908587 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:51:45.908718 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:51:45.908840 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0924 13:51:45.909017 (+ 177us) server_negotiation.cc:299] Negotiation successful
0924 13:51:45.909084 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260924 13:51:45.911306 2828 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a4e07ee1cf9b4d69a506cc049b141c54: Generated new TSK 0
I20260924 13:51:45.911520 2828 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260924 13:51:45.916736 13911 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJN9WRF/build/release/bin/kudu with pid 2748
W20260924 13:51:45.920297 2733 connection.cc:535] client connection to 127.0.0.1:38919 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:38919): Connection reset by peer (error 104)
2026-09-24T13:51:45Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1242 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790257629041645-13911-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 13:51:45 dist-test-slave-sjl2 krb5kdc[2847](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 13:51:45 dist-test-slave-sjl2 krb5kdc[2847](info): set up 2 sockets
Sep 24 13:51:45 dist-test-slave-sjl2 krb5kdc[2847](info): commencing operation
krb5kdc: starting...
W20260924 13:51:46.866017 13911 mini_kdc.cc:121] Time spent starting KDC: real 0.932s user 0.002s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790257629041645-13911-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790257629041645-13911-0/krb5kdc/test-user.keytab.
Sep 24 13:51:46 dist-test-slave-sjl2 krb5kdc[2847](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257906, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T13:51:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T13:51:46Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790257629041645-13911-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790257629041645-13911-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260924 13:51:46.983078 13911 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJN9WRF/build/release/bin/kudu
/tmp/dist-test-taskJN9WRF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790257629041645-13911-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790257629041645-13911-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790257629041645-13911-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790257629041645-13911-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36669
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790257629041645-13911-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42127
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36669
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790257629041645-13911-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790257629041645-13911-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790257629041645-13911-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 13:51:47.062446 2860 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 13:51:47.062662 2860 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 13:51:47.062685 2860 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 13:51:47.063815 2860 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 13:51:47.063854 2860 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 13:51:47.063866 2860 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 13:51:47.063877 2860 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 13:51:47.063889 2860 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 13:51:47.065354 2860 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42127
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790257629041645-13911-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790257629041645-13911-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36669
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790257629041645-13911-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36669
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790257629041645-13911-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2860
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790257629041645-13911-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 12:29:37 UTC on ccycloud.quasar-iqzbxn.root.comops.site
build id 4670106
I20260924 13:51:47.065550 2860 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 13:51:47.065728 2860 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 13:51:47.070968 2866 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 13:51:47.070964 2865 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 13:51:47.070956 2868 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 13:51:47.070986 2860 server_base.cc:1034] running on GCE node
I20260924 13:51:47.071388 2860 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 13:51:47.071661 2860 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 13:51:47.072800 2860 hybrid_clock.cc:648] HybridClock initialized: now 1790257907072788 us; error 29 us; skew 500 ppm
Sep 24 13:51:47 dist-test-slave-sjl2 krb5kdc[2847](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257907, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 13:51:47.075214 2860 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260924 13:51:47.075606 2860 webserver.cc:460] Webserver started at http://127.0.0.1:46405/ using document root <none> and password file <none>
I20260924 13:51:47.075793 2860 fs_manager.cc:359] Metadata directory not provided
I20260924 13:51:47.075832 2860 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 13:51:47.075922 2860 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 13:51:47.076965 2860 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790257629041645-13911-0/minicluster-data/master-0/data/instance:
uuid: "74f0cd92c2bb46c993850d77bafb4dca"
format_stamp: "Formatted at 2026-09-24 13:51:47 on dist-test-slave-sjl2"
server_key: "ac7ab50f007bae76d6bd8f7141054c59"
server_key_iv: "8649bc4a44d1a709f632174d21ea25d6"
server_key_version: "encryptionkey@0"
I20260924 13:51:47.077282 2860 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790257629041645-13911-0/minicluster-data/master-0/wal/instance:
uuid: "74f0cd92c2bb46c993850d77bafb4dca"
format_stamp: "Formatted at 2026-09-24 13:51:47 on dist-test-slave-sjl2"
server_key: "ac7ab50f007bae76d6bd8f7141054c59"
server_key_iv: "8649bc4a44d1a709f632174d21ea25d6"
server_key_version: "encryptionkey@0"
I20260924 13:51:47.078478 2860 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260924 13:51:47.079149 2875 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 13:51:47.079352 2860 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 13:51:47.079424 2860 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790257629041645-13911-0/minicluster-data/master-0/data,/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790257629041645-13911-0/minicluster-data/master-0/wal
uuid: "74f0cd92c2bb46c993850d77bafb4dca"
format_stamp: "Formatted at 2026-09-24 13:51:47 on dist-test-slave-sjl2"
server_key: "ac7ab50f007bae76d6bd8f7141054c59"
server_key_iv: "8649bc4a44d1a709f632174d21ea25d6"
server_key_version: "encryptionkey@0"
I20260924 13:51:47.079480 2860 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790257629041645-13911-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790257629041645-13911-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790257629041645-13911-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 13:51:47.104508 2860 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 13:51:47.106598 2860 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 13:51:47.106716 2860 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 13:51:47.110355 2860 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36669
I20260924 13:51:47.110401 2927 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36669 every 8 connection(s)
I20260924 13:51:47.110718 2860 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790257629041645-13911-0/minicluster-data/master-0/data/info.pb
I20260924 13:51:47.111579 2928 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 13:51:47.113878 2928 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 74f0cd92c2bb46c993850d77bafb4dca: Bootstrap starting.
I20260924 13:51:47.114470 2928 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 74f0cd92c2bb46c993850d77bafb4dca: Neither blocks nor log segments found. Creating new log.
I20260924 13:51:47.114732 2928 log.cc:824] T 00000000000000000000000000000000 P 74f0cd92c2bb46c993850d77bafb4dca: Log is configured to *not* fsync() on all Append() calls
I20260924 13:51:47.115469 2928 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 74f0cd92c2bb46c993850d77bafb4dca: No bootstrap required, opened a new log
I20260924 13:51:47.117079 2928 raft_consensus.cc:348] T 00000000000000000000000000000000 P 74f0cd92c2bb46c993850d77bafb4dca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74f0cd92c2bb46c993850d77bafb4dca" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36669 } }
I20260924 13:51:47.117246 2928 raft_consensus.cc:374] T 00000000000000000000000000000000 P 74f0cd92c2bb46c993850d77bafb4dca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 13:51:47.117282 2928 raft_consensus.cc:729] T 00000000000000000000000000000000 P 74f0cd92c2bb46c993850d77bafb4dca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 74f0cd92c2bb46c993850d77bafb4dca, State: Initialized, Role: FOLLOWER
I20260924 13:51:47.117424 2928 consensus_queue.cc:260] T 00000000000000000000000000000000 P 74f0cd92c2bb46c993850d77bafb4dca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74f0cd92c2bb46c993850d77bafb4dca" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36669 } }
I20260924 13:51:47.117487 2928 raft_consensus.cc:388] T 00000000000000000000000000000000 P 74f0cd92c2bb46c993850d77bafb4dca [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 13:51:47.117517 2928 raft_consensus.cc:482] T 00000000000000000000000000000000 P 74f0cd92c2bb46c993850d77bafb4dca [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 13:51:47.117563 2928 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 74f0cd92c2bb46c993850d77bafb4dca [term 0 FOLLOWER]: Advancing to term 1
I20260924 13:51:47.118151 2928 raft_consensus.cc:504] T 00000000000000000000000000000000 P 74f0cd92c2bb46c993850d77bafb4dca [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74f0cd92c2bb46c993850d77bafb4dca" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36669 } }
I20260924 13:51:47.118286 2928 leader_election.cc:302] T 00000000000000000000000000000000 P 74f0cd92c2bb46c993850d77bafb4dca [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 74f0cd92c2bb46c993850d77bafb4dca; no voters:
I20260924 13:51:47.118301 13911 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJN9WRF/build/release/bin/kudu as pid 2860
I20260924 13:51:47.118415 13911 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790257629041645-13911-0/minicluster-data/master-0/wal/instance
I20260924 13:51:47.118479 2928 leader_election.cc:288] T 00000000000000000000000000000000 P 74f0cd92c2bb46c993850d77bafb4dca [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 13:51:47.118535 2931 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 74f0cd92c2bb46c993850d77bafb4dca [term 1 FOLLOWER]: Leader election won for term 1
I20260924 13:51:47.118600 13911 external_mini_cluster.cc:1428] Setting key 86509f252a51845cfc97a55b6b2f6673
I20260924 13:51:47.118738 2931 raft_consensus.cc:686] T 00000000000000000000000000000000 P 74f0cd92c2bb46c993850d77bafb4dca [term 1 LEADER]: Becoming Leader. State: Replica: 74f0cd92c2bb46c993850d77bafb4dca, State: Running, Role: LEADER
I20260924 13:51:47.118747 2928 sys_catalog.cc:564] T 00000000000000000000000000000000 P 74f0cd92c2bb46c993850d77bafb4dca [sys.catalog]: configured and running, proceeding with master startup.
I20260924 13:51:47.118871 2931 consensus_queue.cc:237] T 00000000000000000000000000000000 P 74f0cd92c2bb46c993850d77bafb4dca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74f0cd92c2bb46c993850d77bafb4dca" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36669 } }
I20260924 13:51:47.120551 2932 sys_catalog.cc:455] T 00000000000000000000000000000000 P 74f0cd92c2bb46c993850d77bafb4dca [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "74f0cd92c2bb46c993850d77bafb4dca" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74f0cd92c2bb46c993850d77bafb4dca" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36669 } } }
I20260924 13:51:47.121030 2932 sys_catalog.cc:458] T 00000000000000000000000000000000 P 74f0cd92c2bb46c993850d77bafb4dca [sys.catalog]: This master's current role is: LEADER
I20260924 13:51:47.120548 2933 sys_catalog.cc:455] T 00000000000000000000000000000000 P 74f0cd92c2bb46c993850d77bafb4dca [sys.catalog]: SysCatalogTable state changed. Reason: New leader 74f0cd92c2bb46c993850d77bafb4dca. Latest consensus state: current_term: 1 leader_uuid: "74f0cd92c2bb46c993850d77bafb4dca" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74f0cd92c2bb46c993850d77bafb4dca" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36669 } } }
I20260924 13:51:47.121258 2933 sys_catalog.cc:458] T 00000000000000000000000000000000 P 74f0cd92c2bb46c993850d77bafb4dca [sys.catalog]: This master's current role is: LEADER
W20260924 13:51:47.121619 2947 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 74f0cd92c2bb46c993850d77bafb4dca: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260924 13:51:47.121662 2947 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260924 13:51:47.121805 2949 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 13:51:47.121944 2949 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 13:51:47.123592 2949 catalog_manager.cc:1347] Generated new cluster ID: a3f06bcac3964bb1bfe7d12734d46f2e
I20260924 13:51:47.123637 2949 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 13:51:47.124538 2935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:47.119080 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.210:57641 (local address 127.0.0.1:36669)
0924 13:51:47.119262 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0924 13:51:47.119273 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0924 13:51:47.119311 (+ 38us) server_negotiation.cc:407] Connection header received
0924 13:51:47.119456 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:51:47.119465 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:51:47.119530 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:51:47.119664 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:51:47.119848 (+ 184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:51:47.121255 (+ 1407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:51:47.122408 (+ 1153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:51:47.122901 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:51:47.123008 (+ 107us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:51:47.123742 (+ 734us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:51:47.123746 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:51:47.123746 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 13:51:47.123960 (+ 214us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:51:47.123999 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 13:51:47.124014 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:51:47.124096 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0924 13:51:47.124288 (+ 192us) server_negotiation.cc:299] Negotiation successful
0924 13:51:47.124318 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124,"thread_start_us":77,"threads_started":1}
I20260924 13:51:47.125464 13911 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260924 13:51:47.130424 2935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:47.126826 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.210:33611 (local address 127.0.0.1:36669)
0924 13:51:47.126859 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0924 13:51:47.126861 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:51:47.127132 (+ 271us) server_negotiation.cc:407] Connection header received
0924 13:51:47.127163 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:51:47.127164 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:51:47.127175 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:51:47.127240 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:51:47.127361 (+ 121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:51:47.128130 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:51:47.129096 (+ 966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:51:47.129419 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:51:47.129489 (+ 70us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:51:47.130187 (+ 698us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:51:47.130189 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:51:47.130189 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 13:51:47.130193 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:51:47.130216 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 13:51:47.130220 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:51:47.130268 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0924 13:51:47.130336 (+ 68us) server_negotiation.cc:299] Negotiation successful
0924 13:51:47.130347 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260924 13:51:47.130735 2949 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 13:51:47.131273 2949 catalog_manager.cc:1504] Loading token signing keys...
I20260924 13:51:47.138530 2949 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 74f0cd92c2bb46c993850d77bafb4dca: Generated new TSK 0
I20260924 13:51:47.138759 2949 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260924 13:51:47.142573 13911 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJN9WRF/build/release/bin/kudu with pid 2860
W20260924 13:51:47.146271 2845 connection.cc:535] client connection to 127.0.0.1:36669 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:36669): Connection reset by peer (error 104)
2026-09-24T13:51:47Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1225 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790257629041645-13911-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 13:51:47 dist-test-slave-sjl2 krb5kdc[2959](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 13:51:47 dist-test-slave-sjl2 krb5kdc[2959](info): set up 2 sockets
Sep 24 13:51:47 dist-test-slave-sjl2 krb5kdc[2959](info): commencing operation
krb5kdc: starting...
W20260924 13:51:48.089963 13911 mini_kdc.cc:121] Time spent starting KDC: real 0.930s user 0.001s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790257629041645-13911-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790257629041645-13911-0/krb5kdc/test-user.keytab.
Sep 24 13:51:48 dist-test-slave-sjl2 krb5kdc[2959](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257908, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T13:51:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T13:51:48Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790257629041645-13911-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790257629041645-13911-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260924 13:51:48.207988 13911 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJN9WRF/build/release/bin/kudu
/tmp/dist-test-taskJN9WRF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790257629041645-13911-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790257629041645-13911-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790257629041645-13911-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790257629041645-13911-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42229
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790257629041645-13911-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45191
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42229
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790257629041645-13911-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790257629041645-13911-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790257629041645-13911-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 13:51:48.288029 2972 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 13:51:48.288239 2972 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 13:51:48.288260 2972 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 13:51:48.289476 2972 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 13:51:48.289525 2972 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 13:51:48.289537 2972 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 13:51:48.289548 2972 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 13:51:48.289558 2972 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 13:51:48.291090 2972 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45191
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790257629041645-13911-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790257629041645-13911-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42229
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790257629041645-13911-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42229
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790257629041645-13911-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2972
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790257629041645-13911-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 12:29:37 UTC on ccycloud.quasar-iqzbxn.root.comops.site
build id 4670106
I20260924 13:51:48.291320 2972 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 13:51:48.291533 2972 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 13:51:48.296599 2977 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 13:51:48.296568 2978 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 13:51:48.296568 2980 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 13:51:48.296618 2972 server_base.cc:1034] running on GCE node
I20260924 13:51:48.297142 2972 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 13:51:48.297427 2972 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 13:51:48.298574 2972 hybrid_clock.cc:648] HybridClock initialized: now 1790257908298562 us; error 33 us; skew 500 ppm
Sep 24 13:51:48 dist-test-slave-sjl2 krb5kdc[2959](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257908, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 13:51:48.301062 2972 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260924 13:51:48.301514 2972 webserver.cc:460] Webserver started at http://127.0.0.1:35779/ using document root <none> and password file <none>
I20260924 13:51:48.301749 2972 fs_manager.cc:359] Metadata directory not provided
I20260924 13:51:48.301795 2972 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 13:51:48.301909 2972 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 13:51:48.302959 2972 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790257629041645-13911-0/minicluster-data/master-0/data/instance:
uuid: "84f567a402d54aee960c587e22f337d7"
format_stamp: "Formatted at 2026-09-24 13:51:48 on dist-test-slave-sjl2"
server_key: "8f78b3ea35d923ad45015d799fd1b2aa"
server_key_iv: "8c96f12c09ded34f2ad00430a571aa55"
server_key_version: "encryptionkey@0"
I20260924 13:51:48.303262 2972 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790257629041645-13911-0/minicluster-data/master-0/wal/instance:
uuid: "84f567a402d54aee960c587e22f337d7"
format_stamp: "Formatted at 2026-09-24 13:51:48 on dist-test-slave-sjl2"
server_key: "8f78b3ea35d923ad45015d799fd1b2aa"
server_key_iv: "8c96f12c09ded34f2ad00430a571aa55"
server_key_version: "encryptionkey@0"
I20260924 13:51:48.304507 2972 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260924 13:51:48.305272 2987 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 13:51:48.305429 2972 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260924 13:51:48.305506 2972 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790257629041645-13911-0/minicluster-data/master-0/data,/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790257629041645-13911-0/minicluster-data/master-0/wal
uuid: "84f567a402d54aee960c587e22f337d7"
format_stamp: "Formatted at 2026-09-24 13:51:48 on dist-test-slave-sjl2"
server_key: "8f78b3ea35d923ad45015d799fd1b2aa"
server_key_iv: "8c96f12c09ded34f2ad00430a571aa55"
server_key_version: "encryptionkey@0"
I20260924 13:51:48.305572 2972 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790257629041645-13911-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790257629041645-13911-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790257629041645-13911-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 13:51:48.308828 2972 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 13:51:48.310988 2972 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 13:51:48.311110 2972 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 13:51:48.314850 2972 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42229
I20260924 13:51:48.314884 3039 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42229 every 8 connection(s)
I20260924 13:51:48.315192 2972 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790257629041645-13911-0/minicluster-data/master-0/data/info.pb
I20260924 13:51:48.316030 3040 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 13:51:48.318423 3040 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 84f567a402d54aee960c587e22f337d7: Bootstrap starting.
I20260924 13:51:48.319013 3040 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 84f567a402d54aee960c587e22f337d7: Neither blocks nor log segments found. Creating new log.
I20260924 13:51:48.319239 3040 log.cc:824] T 00000000000000000000000000000000 P 84f567a402d54aee960c587e22f337d7: Log is configured to *not* fsync() on all Append() calls
I20260924 13:51:48.319900 3040 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 84f567a402d54aee960c587e22f337d7: No bootstrap required, opened a new log
I20260924 13:51:48.321525 3040 raft_consensus.cc:348] T 00000000000000000000000000000000 P 84f567a402d54aee960c587e22f337d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "84f567a402d54aee960c587e22f337d7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42229 } }
I20260924 13:51:48.321674 3040 raft_consensus.cc:374] T 00000000000000000000000000000000 P 84f567a402d54aee960c587e22f337d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 13:51:48.321695 3040 raft_consensus.cc:729] T 00000000000000000000000000000000 P 84f567a402d54aee960c587e22f337d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84f567a402d54aee960c587e22f337d7, State: Initialized, Role: FOLLOWER
I20260924 13:51:48.321815 3040 consensus_queue.cc:260] T 00000000000000000000000000000000 P 84f567a402d54aee960c587e22f337d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "84f567a402d54aee960c587e22f337d7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42229 } }
I20260924 13:51:48.321890 3040 raft_consensus.cc:388] T 00000000000000000000000000000000 P 84f567a402d54aee960c587e22f337d7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 13:51:48.321913 3040 raft_consensus.cc:482] T 00000000000000000000000000000000 P 84f567a402d54aee960c587e22f337d7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 13:51:48.321940 3040 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 84f567a402d54aee960c587e22f337d7 [term 0 FOLLOWER]: Advancing to term 1
I20260924 13:51:48.322444 3040 raft_consensus.cc:504] T 00000000000000000000000000000000 P 84f567a402d54aee960c587e22f337d7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "84f567a402d54aee960c587e22f337d7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42229 } }
I20260924 13:51:48.322531 3040 leader_election.cc:302] T 00000000000000000000000000000000 P 84f567a402d54aee960c587e22f337d7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 84f567a402d54aee960c587e22f337d7; no voters:
I20260924 13:51:48.322743 3040 leader_election.cc:288] T 00000000000000000000000000000000 P 84f567a402d54aee960c587e22f337d7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 13:51:48.322746 13911 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJN9WRF/build/release/bin/kudu as pid 2972
I20260924 13:51:48.322813 3043 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 84f567a402d54aee960c587e22f337d7 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 13:51:48.322862 13911 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790257629041645-13911-0/minicluster-data/master-0/wal/instance
I20260924 13:51:48.323029 13911 external_mini_cluster.cc:1428] Setting key a55299c01ff309876f2b7753b5fb9880
I20260924 13:51:48.323014 3043 raft_consensus.cc:686] T 00000000000000000000000000000000 P 84f567a402d54aee960c587e22f337d7 [term 1 LEADER]: Becoming Leader. State: Replica: 84f567a402d54aee960c587e22f337d7, State: Running, Role: LEADER
I20260924 13:51:48.323025 3040 sys_catalog.cc:564] T 00000000000000000000000000000000 P 84f567a402d54aee960c587e22f337d7 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 13:51:48.323113 3043 consensus_queue.cc:237] T 00000000000000000000000000000000 P 84f567a402d54aee960c587e22f337d7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "84f567a402d54aee960c587e22f337d7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42229 } }
I20260924 13:51:48.323459 3045 sys_catalog.cc:455] T 00000000000000000000000000000000 P 84f567a402d54aee960c587e22f337d7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "84f567a402d54aee960c587e22f337d7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "84f567a402d54aee960c587e22f337d7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42229 } } }
I20260924 13:51:48.323530 3045 sys_catalog.cc:458] T 00000000000000000000000000000000 P 84f567a402d54aee960c587e22f337d7 [sys.catalog]: This master's current role is: LEADER
I20260924 13:51:48.323745 3044 sys_catalog.cc:455] T 00000000000000000000000000000000 P 84f567a402d54aee960c587e22f337d7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 84f567a402d54aee960c587e22f337d7. Latest consensus state: current_term: 1 leader_uuid: "84f567a402d54aee960c587e22f337d7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "84f567a402d54aee960c587e22f337d7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42229 } } }
I20260924 13:51:48.324041 3044 sys_catalog.cc:458] T 00000000000000000000000000000000 P 84f567a402d54aee960c587e22f337d7 [sys.catalog]: This master's current role is: LEADER
I20260924 13:51:48.324064 3048 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 13:51:48.324646 3048 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 13:51:48.326494 3048 catalog_manager.cc:1347] Generated new cluster ID: 0f1c3ab02bce4a15ab6cf4460403596f
I20260924 13:51:48.326556 3048 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 13:51:48.326993 3051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:48.323836 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.210:51351 (local address 127.0.0.1:42229)
0924 13:51:48.325524 (+ 1688us) server_negotiation.cc:206] Beginning negotiation
0924 13:51:48.325532 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0924 13:51:48.325555 (+ 23us) server_negotiation.cc:407] Connection header received
0924 13:51:48.325646 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:51:48.325652 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:51:48.325683 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:51:48.325813 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:51:48.326553 (+ 740us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:51:48.326557 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:51:48.326557 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 13:51:48.326561 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:51:48.326604 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 13:51:48.326619 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:51:48.326656 (+ 37us) server_negotiation.cc:1035] Waiting for connection context
0924 13:51:48.326787 (+ 131us) server_negotiation.cc:299] Negotiation successful
0924 13:51:48.326830 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1640,"thread_start_us":61,"threads_started":1}
I20260924 13:51:48.327890 13911 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260924 13:51:48.330374 3051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:48.329349 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.210:51463 (local address 127.0.0.1:42229)
0924 13:51:48.329380 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0924 13:51:48.329381 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0924 13:51:48.329503 (+ 122us) server_negotiation.cc:407] Connection header received
0924 13:51:48.329565 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:51:48.329566 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:51:48.329570 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:51:48.329618 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:51:48.330181 (+ 563us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:51:48.330182 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:51:48.330182 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 13:51:48.330186 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:51:48.330199 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 13:51:48.330203 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:51:48.330250 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0924 13:51:48.330302 (+ 52us) server_negotiation.cc:299] Negotiation successful
0924 13:51:48.330314 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260924 13:51:48.337693 3048 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 13:51:48.337877 3048 catalog_manager.cc:1504] Loading token signing keys...
I20260924 13:51:48.346560 3048 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 84f567a402d54aee960c587e22f337d7: Generated new TSK 0
I20260924 13:51:48.346776 3048 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260924 13:51:48.354115 13911 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJN9WRF/build/release/bin/kudu with pid 2972
2026-09-24T13:51:48Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1213 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790257629041645-13911-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 13:51:48 dist-test-slave-sjl2 krb5kdc[3071](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 13:51:48 dist-test-slave-sjl2 krb5kdc[3071](info): set up 2 sockets
Sep 24 13:51:48 dist-test-slave-sjl2 krb5kdc[3071](info): commencing operation
krb5kdc: starting...
W20260924 13:51:49.313962 13911 mini_kdc.cc:121] Time spent starting KDC: real 0.941s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790257629041645-13911-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790257629041645-13911-0/krb5kdc/test-user.keytab.
Sep 24 13:51:49 dist-test-slave-sjl2 krb5kdc[3071](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257909, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T13:51:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T13:51:49Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.149.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.149.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.149.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.254.keytab.
Entry for principal kudu/127.13.149.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.254.keytab.
I20260924 13:51:49.432240 13911 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJN9WRF/build/release/bin/kudu
/tmp/dist-test-taskJN9WRF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790257629041645-13911-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790257629041645-13911-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790257629041645-13911-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790257629041645-13911-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.149.254:38793
--webserver_interface=127.13.149.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.254.keytab
--principal=kudu/127.13.149.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.149.212:38447
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.149.254:38793
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790257629041645-13911-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790257629041645-13911-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790257629041645-13911-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 13:51:49.515645 3084 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 13:51:49.515869 3084 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 13:51:49.515901 3084 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 13:51:49.517135 3084 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 13:51:49.517180 3084 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 13:51:49.517199 3084 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 13:51:49.517215 3084 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 13:51:49.517230 3084 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 13:51:49.518718 3084 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.149.212:38447
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790257629041645-13911-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790257629041645-13911-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.149.254:38793
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.254.keytab
--principal=kudu/127.13.149.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.149.254:38793
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790257629041645-13911-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.149.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3084
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790257629041645-13911-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 12:29:37 UTC on ccycloud.quasar-iqzbxn.root.comops.site
build id 4670106
I20260924 13:51:49.518970 3084 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 13:51:49.519177 3084 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 13:51:49.523962 3090 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 13:51:49.523976 3089 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 13:51:49.524003 3092 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 13:51:49.524144 3084 server_base.cc:1034] running on GCE node
I20260924 13:51:49.524506 3084 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 13:51:49.524765 3084 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 13:51:49.525898 3084 hybrid_clock.cc:648] HybridClock initialized: now 1790257909525883 us; error 32 us; skew 500 ppm
Sep 24 13:51:49 dist-test-slave-sjl2 krb5kdc[3071](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257909, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.149.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 13:51:49.532380 3084 init.cc:377] Logged in from keytab as kudu/127.13.149.254@KRBTEST.COM (short username kudu)
I20260924 13:51:49.532874 3084 webserver.cc:460] Webserver started at http://127.13.149.254:43565/ using document root <none> and password file <none>
I20260924 13:51:49.533119 3084 fs_manager.cc:359] Metadata directory not provided
I20260924 13:51:49.533171 3084 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 13:51:49.533306 3084 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 13:51:49.534349 3084 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790257629041645-13911-0/minicluster-data/master-0/data/instance:
uuid: "89978028e03d42a2bbbf2ccffca63089"
format_stamp: "Formatted at 2026-09-24 13:51:49 on dist-test-slave-sjl2"
server_key: "c7d14d42c33a9c94e9591fbed5c66a03"
server_key_iv: "fefa553e404b655b73d00b0c981c5048"
server_key_version: "encryptionkey@0"
I20260924 13:51:49.534678 3084 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790257629041645-13911-0/minicluster-data/master-0/wal/instance:
uuid: "89978028e03d42a2bbbf2ccffca63089"
format_stamp: "Formatted at 2026-09-24 13:51:49 on dist-test-slave-sjl2"
server_key: "c7d14d42c33a9c94e9591fbed5c66a03"
server_key_iv: "fefa553e404b655b73d00b0c981c5048"
server_key_version: "encryptionkey@0"
I20260924 13:51:49.535993 3084 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260924 13:51:49.536689 3099 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 13:51:49.536823 3084 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260924 13:51:49.536906 3084 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790257629041645-13911-0/minicluster-data/master-0/data,/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790257629041645-13911-0/minicluster-data/master-0/wal
uuid: "89978028e03d42a2bbbf2ccffca63089"
format_stamp: "Formatted at 2026-09-24 13:51:49 on dist-test-slave-sjl2"
server_key: "c7d14d42c33a9c94e9591fbed5c66a03"
server_key_iv: "fefa553e404b655b73d00b0c981c5048"
server_key_version: "encryptionkey@0"
I20260924 13:51:49.537005 3084 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790257629041645-13911-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790257629041645-13911-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790257629041645-13911-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 13:51:49.549335 3084 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 13:51:49.551273 3084 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 13:51:49.551419 3084 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 13:51:49.555529 3084 rpc_server.cc:307] RPC server started. Bound to: 127.13.149.254:38793
I20260924 13:51:49.555624 3151 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.149.254:38793 every 8 connection(s)
I20260924 13:51:49.555986 3084 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790257629041645-13911-0/minicluster-data/master-0/data/info.pb
I20260924 13:51:49.556946 3152 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 13:51:49.557103 13911 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJN9WRF/build/release/bin/kudu as pid 3084
I20260924 13:51:49.557268 13911 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790257629041645-13911-0/minicluster-data/master-0/wal/instance
I20260924 13:51:49.557439 13911 external_mini_cluster.cc:1428] Setting key edfb6768e910b6bec3733594ffec4029
I20260924 13:51:49.560230 3152 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 89978028e03d42a2bbbf2ccffca63089: Bootstrap starting.
I20260924 13:51:49.561050 3152 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 89978028e03d42a2bbbf2ccffca63089: Neither blocks nor log segments found. Creating new log.
I20260924 13:51:49.561384 3152 log.cc:824] T 00000000000000000000000000000000 P 89978028e03d42a2bbbf2ccffca63089: Log is configured to *not* fsync() on all Append() calls
I20260924 13:51:49.562350 3152 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 89978028e03d42a2bbbf2ccffca63089: No bootstrap required, opened a new log
I20260924 13:51:49.564821 3152 raft_consensus.cc:348] T 00000000000000000000000000000000 P 89978028e03d42a2bbbf2ccffca63089 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89978028e03d42a2bbbf2ccffca63089" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 38793 } }
I20260924 13:51:49.565079 3152 raft_consensus.cc:374] T 00000000000000000000000000000000 P 89978028e03d42a2bbbf2ccffca63089 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 13:51:49.565158 3152 raft_consensus.cc:729] T 00000000000000000000000000000000 P 89978028e03d42a2bbbf2ccffca63089 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 89978028e03d42a2bbbf2ccffca63089, State: Initialized, Role: FOLLOWER
Sep 24 13:51:49 dist-test-slave-sjl2 krb5kdc[3071](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257909, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.149.254@KRBTEST.COM
I20260924 13:51:49.565308 3152 consensus_queue.cc:260] T 00000000000000000000000000000000 P 89978028e03d42a2bbbf2ccffca63089 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89978028e03d42a2bbbf2ccffca63089" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 38793 } }
I20260924 13:51:49.565407 3152 raft_consensus.cc:388] T 00000000000000000000000000000000 P 89978028e03d42a2bbbf2ccffca63089 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 13:51:49.565450 3152 raft_consensus.cc:482] T 00000000000000000000000000000000 P 89978028e03d42a2bbbf2ccffca63089 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 13:51:49.565500 3152 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 89978028e03d42a2bbbf2ccffca63089 [term 0 FOLLOWER]: Advancing to term 1
I20260924 13:51:49.566246 3152 raft_consensus.cc:504] T 00000000000000000000000000000000 P 89978028e03d42a2bbbf2ccffca63089 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89978028e03d42a2bbbf2ccffca63089" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 38793 } }
I20260924 13:51:49.566354 3152 leader_election.cc:302] T 00000000000000000000000000000000 P 89978028e03d42a2bbbf2ccffca63089 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 89978028e03d42a2bbbf2ccffca63089; no voters:
I20260924 13:51:49.566532 3152 leader_election.cc:288] T 00000000000000000000000000000000 P 89978028e03d42a2bbbf2ccffca63089 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 13:51:49.566584 3157 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 89978028e03d42a2bbbf2ccffca63089 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 13:51:49.566730 3157 raft_consensus.cc:686] T 00000000000000000000000000000000 P 89978028e03d42a2bbbf2ccffca63089 [term 1 LEADER]: Becoming Leader. State: Replica: 89978028e03d42a2bbbf2ccffca63089, State: Running, Role: LEADER
I20260924 13:51:49.566823 3152 sys_catalog.cc:564] T 00000000000000000000000000000000 P 89978028e03d42a2bbbf2ccffca63089 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 13:51:49.566859 3157 consensus_queue.cc:237] T 00000000000000000000000000000000 P 89978028e03d42a2bbbf2ccffca63089 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89978028e03d42a2bbbf2ccffca63089" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 38793 } }
I20260924 13:51:49.567189 3157 sys_catalog.cc:455] T 00000000000000000000000000000000 P 89978028e03d42a2bbbf2ccffca63089 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 89978028e03d42a2bbbf2ccffca63089. Latest consensus state: current_term: 1 leader_uuid: "89978028e03d42a2bbbf2ccffca63089" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89978028e03d42a2bbbf2ccffca63089" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 38793 } } }
I20260924 13:51:49.567266 3157 sys_catalog.cc:458] T 00000000000000000000000000000000 P 89978028e03d42a2bbbf2ccffca63089 [sys.catalog]: This master's current role is: LEADER
I20260924 13:51:49.567373 3158 sys_catalog.cc:455] T 00000000000000000000000000000000 P 89978028e03d42a2bbbf2ccffca63089 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "89978028e03d42a2bbbf2ccffca63089" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89978028e03d42a2bbbf2ccffca63089" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 38793 } } }
I20260924 13:51:49.567510 3158 sys_catalog.cc:458] T 00000000000000000000000000000000 P 89978028e03d42a2bbbf2ccffca63089 [sys.catalog]: This master's current role is: LEADER
I20260924 13:51:49.567919 3163 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 13:51:49.568552 3163 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 13:51:49.570214 3155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:49.558267 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51917 (local address 127.13.149.254:38793)
0924 13:51:49.558473 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0924 13:51:49.558484 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0924 13:51:49.558521 (+ 37us) server_negotiation.cc:407] Connection header received
0924 13:51:49.559051 (+ 530us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:51:49.559062 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:51:49.559128 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:51:49.559300 (+ 172us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:51:49.559971 (+ 671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:51:49.561294 (+ 1323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:51:49.562319 (+ 1025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:51:49.562963 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:51:49.563089 (+ 126us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:51:49.565888 (+ 2799us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:51:49.565893 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:51:49.565894 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 13:51:49.565908 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:51:49.568327 (+ 2419us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:51:49.568891 (+ 564us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:51:49.568895 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:51:49.568895 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:51:49.569036 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:51:49.569297 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:51:49.569298 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:51:49.569298 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:51:49.569610 (+ 312us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:51:49.569817 (+ 207us) server_negotiation.cc:1035] Waiting for connection context
0924 13:51:49.569945 (+ 128us) server_negotiation.cc:299] Negotiation successful
0924 13:51:49.570042 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":154,"thread_start_us":76,"threads_started":1}
I20260924 13:51:49.570675 3163 catalog_manager.cc:1347] Generated new cluster ID: 59d039a8337e40ebb3d5600f7e90205a
I20260924 13:51:49.570720 3163 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 13:51:49.571231 13911 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260924 13:51:49.579950 3155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:49.572723 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34355 (local address 127.13.149.254:38793)
0924 13:51:49.572779 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0924 13:51:49.572781 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:51:49.572879 (+ 98us) server_negotiation.cc:407] Connection header received
0924 13:51:49.572946 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:51:49.572947 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:51:49.572956 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:51:49.573024 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:51:49.573633 (+ 609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:51:49.574387 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:51:49.575368 (+ 981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:51:49.575710 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:51:49.575785 (+ 75us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:51:49.577211 (+ 1426us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:51:49.577213 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:51:49.577214 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 13:51:49.577221 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:51:49.578766 (+ 1545us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:51:49.579173 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:51:49.579175 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:51:49.579175 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:51:49.579237 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:51:49.579429 (+ 192us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:51:49.579430 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:51:49.579430 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:51:49.579566 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:51:49.579671 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0924 13:51:49.579782 (+ 111us) server_negotiation.cc:299] Negotiation successful
0924 13:51:49.579848 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260924 13:51:49.589372 3163 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 13:51:49.590159 3163 catalog_manager.cc:1504] Loading token signing keys...
I20260924 13:51:49.603399 3163 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 89978028e03d42a2bbbf2ccffca63089: Generated new TSK 0
I20260924 13:51:49.603679 3163 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260924 13:51:49.616022 13911 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJN9WRF/build/release/bin/kudu with pid 3084
W20260924 13:51:49.619856 3069 connection.cc:535] client connection to 127.13.149.254:38793 recv error: Network error: failed to read from TLS socket (remote: 127.13.149.254:38793): Connection reset by peer (error 104)
2026-09-24T13:51:49Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1259 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790257629041645-13911-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 13:51:49 dist-test-slave-sjl2 krb5kdc[3183](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 13:51:49 dist-test-slave-sjl2 krb5kdc[3183](info): set up 2 sockets
Sep 24 13:51:49 dist-test-slave-sjl2 krb5kdc[3183](info): commencing operation
krb5kdc: starting...
W20260924 13:51:50.570029 13911 mini_kdc.cc:121] Time spent starting KDC: real 0.937s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790257629041645-13911-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790257629041645-13911-0/krb5kdc/test-user.keytab.
Sep 24 13:51:50 dist-test-slave-sjl2 krb5kdc[3183](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257910, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T13:51:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T13:51:50Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.149.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.149.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.149.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.254.keytab.
Entry for principal kudu/127.13.149.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.254.keytab.
I20260924 13:51:50.688669 13911 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJN9WRF/build/release/bin/kudu
/tmp/dist-test-taskJN9WRF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790257629041645-13911-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790257629041645-13911-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790257629041645-13911-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790257629041645-13911-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.149.254:44121
--webserver_interface=127.13.149.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.254.keytab
--principal=kudu/127.13.149.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.149.212:36525
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.149.254:44121
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790257629041645-13911-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790257629041645-13911-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790257629041645-13911-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 13:51:50.769739 3196 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 13:51:50.769974 3196 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 13:51:50.770008 3196 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 13:51:50.771255 3196 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 13:51:50.771307 3196 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 13:51:50.771325 3196 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 13:51:50.771341 3196 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 13:51:50.771356 3196 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 13:51:50.773021 3196 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.149.212:36525
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790257629041645-13911-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790257629041645-13911-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.149.254:44121
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.254.keytab
--principal=kudu/127.13.149.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.149.254:44121
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790257629041645-13911-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.149.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3196
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790257629041645-13911-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 12:29:37 UTC on ccycloud.quasar-iqzbxn.root.comops.site
build id 4670106
I20260924 13:51:50.773305 3196 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 13:51:50.773525 3196 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260924 13:51:50.778481 3196 server_base.cc:1034] running on GCE node
W20260924 13:51:50.778390 3204 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 13:51:50.778412 3202 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 13:51:50.778393 3201 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 13:51:50.778962 3196 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 13:51:50.779229 3196 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 13:51:50.780368 3196 hybrid_clock.cc:648] HybridClock initialized: now 1790257910780356 us; error 32 us; skew 500 ppm
Sep 24 13:51:50 dist-test-slave-sjl2 krb5kdc[3183](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257910, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.149.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 13:51:50.782843 3196 init.cc:377] Logged in from keytab as kudu/127.13.149.254@KRBTEST.COM (short username kudu)
I20260924 13:51:50.783317 3196 webserver.cc:460] Webserver started at http://127.13.149.254:46401/ using document root <none> and password file <none>
I20260924 13:51:50.783533 3196 fs_manager.cc:359] Metadata directory not provided
I20260924 13:51:50.783581 3196 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 13:51:50.783706 3196 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 13:51:50.784762 3196 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790257629041645-13911-0/minicluster-data/master-0/data/instance:
uuid: "d406783a16564cac834c7afdafd573d9"
format_stamp: "Formatted at 2026-09-24 13:51:50 on dist-test-slave-sjl2"
server_key: "2d6643e13077bc95f25b0ad2c8de7e85"
server_key_iv: "8ec77c527bfe35cca6e976c680efeca7"
server_key_version: "encryptionkey@0"
I20260924 13:51:50.785123 3196 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790257629041645-13911-0/minicluster-data/master-0/wal/instance:
uuid: "d406783a16564cac834c7afdafd573d9"
format_stamp: "Formatted at 2026-09-24 13:51:50 on dist-test-slave-sjl2"
server_key: "2d6643e13077bc95f25b0ad2c8de7e85"
server_key_iv: "8ec77c527bfe35cca6e976c680efeca7"
server_key_version: "encryptionkey@0"
I20260924 13:51:50.786429 3196 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260924 13:51:50.787197 3211 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 13:51:50.787372 3196 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260924 13:51:50.787454 3196 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790257629041645-13911-0/minicluster-data/master-0/data,/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790257629041645-13911-0/minicluster-data/master-0/wal
uuid: "d406783a16564cac834c7afdafd573d9"
format_stamp: "Formatted at 2026-09-24 13:51:50 on dist-test-slave-sjl2"
server_key: "2d6643e13077bc95f25b0ad2c8de7e85"
server_key_iv: "8ec77c527bfe35cca6e976c680efeca7"
server_key_version: "encryptionkey@0"
I20260924 13:51:50.787520 3196 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790257629041645-13911-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790257629041645-13911-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790257629041645-13911-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 13:51:50.802407 3196 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 13:51:50.804659 3196 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 13:51:50.804805 3196 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 13:51:50.808494 3196 rpc_server.cc:307] RPC server started. Bound to: 127.13.149.254:44121
I20260924 13:51:50.808549 3263 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.149.254:44121 every 8 connection(s)
I20260924 13:51:50.808863 3196 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790257629041645-13911-0/minicluster-data/master-0/data/info.pb
I20260924 13:51:50.809719 3264 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 13:51:50.811972 3264 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d406783a16564cac834c7afdafd573d9: Bootstrap starting.
I20260924 13:51:50.812544 3264 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d406783a16564cac834c7afdafd573d9: Neither blocks nor log segments found. Creating new log.
I20260924 13:51:50.812793 3264 log.cc:824] T 00000000000000000000000000000000 P d406783a16564cac834c7afdafd573d9: Log is configured to *not* fsync() on all Append() calls
I20260924 13:51:50.813562 3264 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d406783a16564cac834c7afdafd573d9: No bootstrap required, opened a new log
I20260924 13:51:50.813777 13911 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJN9WRF/build/release/bin/kudu as pid 3196
I20260924 13:51:50.813880 13911 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790257629041645-13911-0/minicluster-data/master-0/wal/instance
I20260924 13:51:50.814029 13911 external_mini_cluster.cc:1428] Setting key 074c69cb1a5d96bfd87120f8e2f454af
I20260924 13:51:50.815701 3264 raft_consensus.cc:348] T 00000000000000000000000000000000 P d406783a16564cac834c7afdafd573d9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d406783a16564cac834c7afdafd573d9" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 44121 } }
I20260924 13:51:50.815861 3264 raft_consensus.cc:374] T 00000000000000000000000000000000 P d406783a16564cac834c7afdafd573d9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 13:51:50.815884 3264 raft_consensus.cc:729] T 00000000000000000000000000000000 P d406783a16564cac834c7afdafd573d9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d406783a16564cac834c7afdafd573d9, State: Initialized, Role: FOLLOWER
I20260924 13:51:50.815990 3264 consensus_queue.cc:260] T 00000000000000000000000000000000 P d406783a16564cac834c7afdafd573d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d406783a16564cac834c7afdafd573d9" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 44121 } }
I20260924 13:51:50.816057 3264 raft_consensus.cc:388] T 00000000000000000000000000000000 P d406783a16564cac834c7afdafd573d9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 13:51:50.816078 3264 raft_consensus.cc:482] T 00000000000000000000000000000000 P d406783a16564cac834c7afdafd573d9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 13:51:50.816124 3264 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d406783a16564cac834c7afdafd573d9 [term 0 FOLLOWER]: Advancing to term 1
I20260924 13:51:50.816707 3264 raft_consensus.cc:504] T 00000000000000000000000000000000 P d406783a16564cac834c7afdafd573d9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d406783a16564cac834c7afdafd573d9" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 44121 } }
I20260924 13:51:50.816799 3264 leader_election.cc:302] T 00000000000000000000000000000000 P d406783a16564cac834c7afdafd573d9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d406783a16564cac834c7afdafd573d9; no voters:
I20260924 13:51:50.817016 3264 leader_election.cc:288] T 00000000000000000000000000000000 P d406783a16564cac834c7afdafd573d9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 13:51:50.817044 3269 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d406783a16564cac834c7afdafd573d9 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 13:51:50.817147 3269 raft_consensus.cc:686] T 00000000000000000000000000000000 P d406783a16564cac834c7afdafd573d9 [term 1 LEADER]: Becoming Leader. State: Replica: d406783a16564cac834c7afdafd573d9, State: Running, Role: LEADER
I20260924 13:51:50.817291 3269 consensus_queue.cc:237] T 00000000000000000000000000000000 P d406783a16564cac834c7afdafd573d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d406783a16564cac834c7afdafd573d9" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 44121 } }
I20260924 13:51:50.817339 3264 sys_catalog.cc:564] T 00000000000000000000000000000000 P d406783a16564cac834c7afdafd573d9 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 13:51:50.817603 3269 sys_catalog.cc:455] T 00000000000000000000000000000000 P d406783a16564cac834c7afdafd573d9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d406783a16564cac834c7afdafd573d9. Latest consensus state: current_term: 1 leader_uuid: "d406783a16564cac834c7afdafd573d9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d406783a16564cac834c7afdafd573d9" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 44121 } } }
I20260924 13:51:50.817826 3269 sys_catalog.cc:458] T 00000000000000000000000000000000 P d406783a16564cac834c7afdafd573d9 [sys.catalog]: This master's current role is: LEADER
I20260924 13:51:50.817760 3270 sys_catalog.cc:455] T 00000000000000000000000000000000 P d406783a16564cac834c7afdafd573d9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d406783a16564cac834c7afdafd573d9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d406783a16564cac834c7afdafd573d9" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 44121 } } }
I20260924 13:51:50.818230 3270 sys_catalog.cc:458] T 00000000000000000000000000000000 P d406783a16564cac834c7afdafd573d9 [sys.catalog]: This master's current role is: LEADER
I20260924 13:51:50.818427 3278 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 13:51:50.818953 3278 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 13:51:50.820550 3268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:50.814518 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51965 (local address 127.13.149.254:44121)
0924 13:51:50.814688 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0924 13:51:50.814696 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0924 13:51:50.814722 (+ 26us) server_negotiation.cc:407] Connection header received
0924 13:51:50.815272 (+ 550us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:51:50.815279 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:51:50.815331 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:51:50.815450 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:51:50.815741 (+ 291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:51:50.817059 (+ 1318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:51:50.818204 (+ 1145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:51:50.818901 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:51:50.819016 (+ 115us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:51:50.819848 (+ 832us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:51:50.819852 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:51:50.819853 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 13:51:50.820063 (+ 210us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:51:50.820103 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 13:51:50.820120 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:51:50.820203 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0924 13:51:50.820343 (+ 140us) server_negotiation.cc:299] Negotiation successful
0924 13:51:50.820373 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":112,"thread_start_us":71,"threads_started":1}
I20260924 13:51:50.820847 3278 catalog_manager.cc:1347] Generated new cluster ID: 9ba5d585e9574fb3be324e6013dfdc57
I20260924 13:51:50.820942 3278 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 13:51:50.821538 13911 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260924 13:51:50.825857 3268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:50.822890 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35789 (local address 127.13.149.254:44121)
0924 13:51:50.822929 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0924 13:51:50.822931 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:51:50.823029 (+ 98us) server_negotiation.cc:407] Connection header received
0924 13:51:50.823090 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:51:50.823091 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:51:50.823100 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:51:50.823146 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:51:50.823282 (+ 136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:51:50.823773 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:51:50.824488 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:51:50.824725 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:51:50.824783 (+ 58us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:51:50.825527 (+ 744us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:51:50.825529 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:51:50.825529 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 13:51:50.825534 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:51:50.825559 (+ 25us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 13:51:50.825564 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:51:50.825612 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0924 13:51:50.825717 (+ 105us) server_negotiation.cc:299] Negotiation successful
0924 13:51:50.825732 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260924 13:51:50.829484 3278 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 13:51:50.830044 3278 catalog_manager.cc:1504] Loading token signing keys...
I20260924 13:51:50.833658 3278 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d406783a16564cac834c7afdafd573d9: Generated new TSK 0
I20260924 13:51:50.833945 3278 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260924 13:51:50.842929 13911 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJN9WRF/build/release/bin/kudu with pid 3196
W20260924 13:51:50.846829 3181 connection.cc:535] client connection to 127.13.149.254:44121 recv error: Network error: failed to read from TLS socket (remote: 127.13.149.254:44121): Connection reset by peer (error 104)
2026-09-24T13:51:50Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1226 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790257629041645-13911-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 13:51:50 dist-test-slave-sjl2 krb5kdc[3295](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 13:51:50 dist-test-slave-sjl2 krb5kdc[3295](info): set up 2 sockets
Sep 24 13:51:50 dist-test-slave-sjl2 krb5kdc[3295](info): commencing operation
krb5kdc: starting...
W20260924 13:51:51.793982 13911 mini_kdc.cc:121] Time spent starting KDC: real 0.934s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790257629041645-13911-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790257629041645-13911-0/krb5kdc/test-user.keytab.
Sep 24 13:51:51 dist-test-slave-sjl2 krb5kdc[3295](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257911, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T13:51:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T13:51:51Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.149.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.149.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.149.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.254.keytab.
Entry for principal kudu/127.13.149.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.254.keytab.
I20260924 13:51:51.912271 13911 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJN9WRF/build/release/bin/kudu
/tmp/dist-test-taskJN9WRF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790257629041645-13911-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790257629041645-13911-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790257629041645-13911-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790257629041645-13911-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.149.254:43399
--webserver_interface=127.13.149.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.254.keytab
--principal=kudu/127.13.149.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.149.212:37881
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.149.254:43399
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790257629041645-13911-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790257629041645-13911-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790257629041645-13911-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 13:51:51.990909 3308 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 13:51:51.991140 3308 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 13:51:51.991164 3308 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 13:51:51.992306 3308 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 13:51:51.992345 3308 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 13:51:51.992358 3308 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 13:51:51.992368 3308 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 13:51:51.992379 3308 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 13:51:51.993911 3308 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.149.212:37881
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790257629041645-13911-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790257629041645-13911-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.149.254:43399
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.254.keytab
--principal=kudu/127.13.149.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.149.254:43399
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790257629041645-13911-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.149.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3308
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790257629041645-13911-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 12:29:37 UTC on ccycloud.quasar-iqzbxn.root.comops.site
build id 4670106
I20260924 13:51:51.994146 3308 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 13:51:51.994323 3308 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260924 13:51:51.999274 3308 server_base.cc:1034] running on GCE node
W20260924 13:51:51.999243 3316 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 13:51:51.999243 3313 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 13:51:51.999248 3314 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 13:51:51.999817 3308 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 13:51:52.000140 3308 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 13:51:52.001281 3308 hybrid_clock.cc:648] HybridClock initialized: now 1790257912001261 us; error 35 us; skew 500 ppm
Sep 24 13:51:52 dist-test-slave-sjl2 krb5kdc[3295](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257912, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.149.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 13:51:52.003831 3308 init.cc:377] Logged in from keytab as kudu/127.13.149.254@KRBTEST.COM (short username kudu)
I20260924 13:51:52.004262 3308 webserver.cc:460] Webserver started at http://127.13.149.254:43497/ using document root <none> and password file <none>
I20260924 13:51:52.004460 3308 fs_manager.cc:359] Metadata directory not provided
I20260924 13:51:52.004508 3308 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 13:51:52.004632 3308 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 13:51:52.005734 3308 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790257629041645-13911-0/minicluster-data/master-0/data/instance:
uuid: "95cd33da34f14bc498e9d490e360bb11"
format_stamp: "Formatted at 2026-09-24 13:51:52 on dist-test-slave-sjl2"
server_key: "c953a769bbb9bc33d6f8ed437d1d6364"
server_key_iv: "2f98b914b620b9effd1ed94f8e9ca351"
server_key_version: "encryptionkey@0"
I20260924 13:51:52.006078 3308 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790257629041645-13911-0/minicluster-data/master-0/wal/instance:
uuid: "95cd33da34f14bc498e9d490e360bb11"
format_stamp: "Formatted at 2026-09-24 13:51:52 on dist-test-slave-sjl2"
server_key: "c953a769bbb9bc33d6f8ed437d1d6364"
server_key_iv: "2f98b914b620b9effd1ed94f8e9ca351"
server_key_version: "encryptionkey@0"
I20260924 13:51:52.007436 3308 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260924 13:51:52.008203 3323 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 13:51:52.008359 3308 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260924 13:51:52.008440 3308 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790257629041645-13911-0/minicluster-data/master-0/data,/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790257629041645-13911-0/minicluster-data/master-0/wal
uuid: "95cd33da34f14bc498e9d490e360bb11"
format_stamp: "Formatted at 2026-09-24 13:51:52 on dist-test-slave-sjl2"
server_key: "c953a769bbb9bc33d6f8ed437d1d6364"
server_key_iv: "2f98b914b620b9effd1ed94f8e9ca351"
server_key_version: "encryptionkey@0"
I20260924 13:51:52.008504 3308 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790257629041645-13911-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790257629041645-13911-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790257629041645-13911-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 13:51:52.011758 3308 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 13:51:52.013866 3308 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 13:51:52.013993 3308 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 13:51:52.017656 3308 rpc_server.cc:307] RPC server started. Bound to: 127.13.149.254:43399
I20260924 13:51:52.017758 3375 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.149.254:43399 every 8 connection(s)
I20260924 13:51:52.018025 3308 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790257629041645-13911-0/minicluster-data/master-0/data/info.pb
I20260924 13:51:52.019032 3376 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 13:51:52.021726 3376 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 95cd33da34f14bc498e9d490e360bb11: Bootstrap starting.
I20260924 13:51:52.022331 3376 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 95cd33da34f14bc498e9d490e360bb11: Neither blocks nor log segments found. Creating new log.
I20260924 13:51:52.022599 3376 log.cc:824] T 00000000000000000000000000000000 P 95cd33da34f14bc498e9d490e360bb11: Log is configured to *not* fsync() on all Append() calls
I20260924 13:51:52.023433 3376 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 95cd33da34f14bc498e9d490e360bb11: No bootstrap required, opened a new log
I20260924 13:51:52.025211 3376 raft_consensus.cc:348] T 00000000000000000000000000000000 P 95cd33da34f14bc498e9d490e360bb11 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "95cd33da34f14bc498e9d490e360bb11" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 43399 } }
I20260924 13:51:52.025369 3376 raft_consensus.cc:374] T 00000000000000000000000000000000 P 95cd33da34f14bc498e9d490e360bb11 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 13:51:52.025393 3376 raft_consensus.cc:729] T 00000000000000000000000000000000 P 95cd33da34f14bc498e9d490e360bb11 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 95cd33da34f14bc498e9d490e360bb11, State: Initialized, Role: FOLLOWER
I20260924 13:51:52.025533 3376 consensus_queue.cc:260] T 00000000000000000000000000000000 P 95cd33da34f14bc498e9d490e360bb11 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "95cd33da34f14bc498e9d490e360bb11" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 43399 } }
I20260924 13:51:52.025611 3376 raft_consensus.cc:388] T 00000000000000000000000000000000 P 95cd33da34f14bc498e9d490e360bb11 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 13:51:52.025636 3376 raft_consensus.cc:482] T 00000000000000000000000000000000 P 95cd33da34f14bc498e9d490e360bb11 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 13:51:52.025667 3376 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 95cd33da34f14bc498e9d490e360bb11 [term 0 FOLLOWER]: Advancing to term 1
I20260924 13:51:52.026568 3376 raft_consensus.cc:504] T 00000000000000000000000000000000 P 95cd33da34f14bc498e9d490e360bb11 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "95cd33da34f14bc498e9d490e360bb11" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 43399 } }
I20260924 13:51:52.026676 3376 leader_election.cc:302] T 00000000000000000000000000000000 P 95cd33da34f14bc498e9d490e360bb11 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 95cd33da34f14bc498e9d490e360bb11; no voters:
I20260924 13:51:52.026690 13911 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJN9WRF/build/release/bin/kudu as pid 3308
I20260924 13:51:52.026785 13911 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790257629041645-13911-0/minicluster-data/master-0/wal/instance
I20260924 13:51:52.026855 3376 leader_election.cc:288] T 00000000000000000000000000000000 P 95cd33da34f14bc498e9d490e360bb11 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 13:51:52.026942 3379 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 95cd33da34f14bc498e9d490e360bb11 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 13:51:52.026979 13911 external_mini_cluster.cc:1428] Setting key e3798d4391939619fcd2c7695737494e
I20260924 13:51:52.027128 3376 sys_catalog.cc:564] T 00000000000000000000000000000000 P 95cd33da34f14bc498e9d490e360bb11 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 13:51:52.027145 3379 raft_consensus.cc:686] T 00000000000000000000000000000000 P 95cd33da34f14bc498e9d490e360bb11 [term 1 LEADER]: Becoming Leader. State: Replica: 95cd33da34f14bc498e9d490e360bb11, State: Running, Role: LEADER
I20260924 13:51:52.027355 3379 consensus_queue.cc:237] T 00000000000000000000000000000000 P 95cd33da34f14bc498e9d490e360bb11 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "95cd33da34f14bc498e9d490e360bb11" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 43399 } }
I20260924 13:51:52.027797 3380 sys_catalog.cc:455] T 00000000000000000000000000000000 P 95cd33da34f14bc498e9d490e360bb11 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "95cd33da34f14bc498e9d490e360bb11" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "95cd33da34f14bc498e9d490e360bb11" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 43399 } } }
I20260924 13:51:52.027814 3381 sys_catalog.cc:455] T 00000000000000000000000000000000 P 95cd33da34f14bc498e9d490e360bb11 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 95cd33da34f14bc498e9d490e360bb11. Latest consensus state: current_term: 1 leader_uuid: "95cd33da34f14bc498e9d490e360bb11" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "95cd33da34f14bc498e9d490e360bb11" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 43399 } } }
I20260924 13:51:52.027976 3381 sys_catalog.cc:458] T 00000000000000000000000000000000 P 95cd33da34f14bc498e9d490e360bb11 [sys.catalog]: This master's current role is: LEADER
I20260924 13:51:52.027966 3380 sys_catalog.cc:458] T 00000000000000000000000000000000 P 95cd33da34f14bc498e9d490e360bb11 [sys.catalog]: This master's current role is: LEADER
I20260924 13:51:52.028390 3390 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 13:51:52.029345 3390 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 13:51:52.030035 3383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:52.027492 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45263 (local address 127.13.149.254:43399)
0924 13:51:52.028358 (+ 866us) server_negotiation.cc:206] Beginning negotiation
0924 13:51:52.028371 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0924 13:51:52.028408 (+ 37us) server_negotiation.cc:407] Connection header received
0924 13:51:52.028543 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:51:52.028551 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:51:52.028587 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:51:52.028777 (+ 190us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:51:52.029493 (+ 716us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:51:52.029497 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:51:52.029497 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 13:51:52.029501 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:51:52.029544 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 13:51:52.029559 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:51:52.029626 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0924 13:51:52.029721 (+ 95us) server_negotiation.cc:299] Negotiation successful
0924 13:51:52.029853 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":804,"thread_start_us":70,"threads_started":1}
I20260924 13:51:52.030929 3390 catalog_manager.cc:1347] Generated new cluster ID: 51c30d5325e645f3a7f0987a50bf683a
I20260924 13:51:52.030937 13911 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260924 13:51:52.030977 3390 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 13:51:52.033571 3383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:52.032562 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51447 (local address 127.13.149.254:43399)
0924 13:51:52.032598 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0924 13:51:52.032599 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0924 13:51:52.032648 (+ 49us) server_negotiation.cc:407] Connection header received
0924 13:51:52.032740 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:51:52.032741 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:51:52.032746 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:51:52.032792 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:51:52.033371 (+ 579us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:51:52.033372 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:51:52.033373 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 13:51:52.033376 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:51:52.033391 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 13:51:52.033395 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:51:52.033441 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0924 13:51:52.033485 (+ 44us) server_negotiation.cc:299] Negotiation successful
0924 13:51:52.033497 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260924 13:51:52.041260 3390 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 13:51:52.041441 3390 catalog_manager.cc:1504] Loading token signing keys...
I20260924 13:51:52.051570 3390 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 95cd33da34f14bc498e9d490e360bb11: Generated new TSK 0
I20260924 13:51:52.051795 3390 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260924 13:51:52.054520 13911 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJN9WRF/build/release/bin/kudu with pid 3308
2026-09-24T13:51:52Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1212 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790257629041645-13911-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 13:51:52 dist-test-slave-sjl2 krb5kdc[3407](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 13:51:52 dist-test-slave-sjl2 krb5kdc[3407](info): set up 2 sockets
Sep 24 13:51:52 dist-test-slave-sjl2 krb5kdc[3407](info): commencing operation
krb5kdc: starting...
W20260924 13:51:53.022046 13911 mini_kdc.cc:121] Time spent starting KDC: real 0.949s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790257629041645-13911-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790257629041645-13911-0/krb5kdc/test-user.keytab.
Sep 24 13:51:53 dist-test-slave-sjl2 krb5kdc[3407](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257913, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T13:51:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T13:51:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.149.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.149.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.149.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.254.keytab.
Entry for principal kudu/127.13.149.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.254.keytab.
I20260924 13:51:53.140776 13911 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJN9WRF/build/release/bin/kudu
/tmp/dist-test-taskJN9WRF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790257629041645-13911-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790257629041645-13911-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790257629041645-13911-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790257629041645-13911-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.149.254:36913
--webserver_interface=127.13.149.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.254.keytab
--principal=kudu/127.13.149.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.149.212:38469
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.149.254:36913
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790257629041645-13911-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790257629041645-13911-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790257629041645-13911-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 13:51:53.223801 3420 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 13:51:53.224040 3420 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 13:51:53.224072 3420 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 13:51:53.225400 3420 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 13:51:53.225452 3420 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 13:51:53.225472 3420 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 13:51:53.225488 3420 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 13:51:53.225504 3420 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 13:51:53.227173 3420 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.149.212:38469
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790257629041645-13911-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790257629041645-13911-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.149.254:36913
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.254.keytab
--principal=kudu/127.13.149.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.149.254:36913
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790257629041645-13911-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.149.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3420
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790257629041645-13911-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 12:29:37 UTC on ccycloud.quasar-iqzbxn.root.comops.site
build id 4670106
I20260924 13:51:53.227450 3420 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 13:51:53.227679 3420 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 13:51:53.232569 3428 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 13:51:53.232632 3426 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 13:51:53.232735 3425 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 13:51:53.232895 3420 server_base.cc:1034] running on GCE node
I20260924 13:51:53.233183 3420 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 13:51:53.233481 3420 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 13:51:53.234622 3420 hybrid_clock.cc:648] HybridClock initialized: now 1790257913234602 us; error 37 us; skew 500 ppm
Sep 24 13:51:53 dist-test-slave-sjl2 krb5kdc[3407](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257913, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.149.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 13:51:53.237187 3420 init.cc:377] Logged in from keytab as kudu/127.13.149.254@KRBTEST.COM (short username kudu)
I20260924 13:51:53.237701 3420 webserver.cc:460] Webserver started at http://127.13.149.254:34755/ using document root <none> and password file <none>
I20260924 13:51:53.237939 3420 fs_manager.cc:359] Metadata directory not provided
I20260924 13:51:53.237988 3420 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 13:51:53.238103 3420 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 13:51:53.239202 3420 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790257629041645-13911-0/minicluster-data/master-0/data/instance:
uuid: "6f1b3118c44c47c0a904409826061aad"
format_stamp: "Formatted at 2026-09-24 13:51:53 on dist-test-slave-sjl2"
server_key: "f2978d9d2cc629bc09fc559471b56207"
server_key_iv: "c68578fb69405ce5a74e038579e06398"
server_key_version: "encryptionkey@0"
I20260924 13:51:53.239560 3420 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790257629041645-13911-0/minicluster-data/master-0/wal/instance:
uuid: "6f1b3118c44c47c0a904409826061aad"
format_stamp: "Formatted at 2026-09-24 13:51:53 on dist-test-slave-sjl2"
server_key: "f2978d9d2cc629bc09fc559471b56207"
server_key_iv: "c68578fb69405ce5a74e038579e06398"
server_key_version: "encryptionkey@0"
I20260924 13:51:53.240911 3420 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260924 13:51:53.241803 3435 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 13:51:53.242024 3420 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260924 13:51:53.242115 3420 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790257629041645-13911-0/minicluster-data/master-0/data,/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790257629041645-13911-0/minicluster-data/master-0/wal
uuid: "6f1b3118c44c47c0a904409826061aad"
format_stamp: "Formatted at 2026-09-24 13:51:53 on dist-test-slave-sjl2"
server_key: "f2978d9d2cc629bc09fc559471b56207"
server_key_iv: "c68578fb69405ce5a74e038579e06398"
server_key_version: "encryptionkey@0"
I20260924 13:51:53.242202 3420 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790257629041645-13911-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790257629041645-13911-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790257629041645-13911-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 13:51:53.265470 3420 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 13:51:53.267761 3420 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 13:51:53.267889 3420 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 13:51:53.271858 3420 rpc_server.cc:307] RPC server started. Bound to: 127.13.149.254:36913
I20260924 13:51:53.272217 3487 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.149.254:36913 every 8 connection(s)
I20260924 13:51:53.272276 3420 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790257629041645-13911-0/minicluster-data/master-0/data/info.pb
I20260924 13:51:53.272994 3488 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 13:51:53.275355 3488 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6f1b3118c44c47c0a904409826061aad: Bootstrap starting.
I20260924 13:51:53.275980 3488 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6f1b3118c44c47c0a904409826061aad: Neither blocks nor log segments found. Creating new log.
I20260924 13:51:53.276116 13911 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJN9WRF/build/release/bin/kudu as pid 3420
I20260924 13:51:53.276228 13911 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790257629041645-13911-0/minicluster-data/master-0/wal/instance
I20260924 13:51:53.276298 3488 log.cc:824] T 00000000000000000000000000000000 P 6f1b3118c44c47c0a904409826061aad: Log is configured to *not* fsync() on all Append() calls
I20260924 13:51:53.276438 13911 external_mini_cluster.cc:1428] Setting key d8bda7b706ec039623d67fbe5b9f482d
I20260924 13:51:53.277206 3488 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6f1b3118c44c47c0a904409826061aad: No bootstrap required, opened a new log
I20260924 13:51:53.279865 3488 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6f1b3118c44c47c0a904409826061aad [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f1b3118c44c47c0a904409826061aad" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 36913 } }
I20260924 13:51:53.280052 3488 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6f1b3118c44c47c0a904409826061aad [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 13:51:53.280092 3488 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6f1b3118c44c47c0a904409826061aad [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f1b3118c44c47c0a904409826061aad, State: Initialized, Role: FOLLOWER
I20260924 13:51:53.280231 3488 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6f1b3118c44c47c0a904409826061aad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f1b3118c44c47c0a904409826061aad" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 36913 } }
I20260924 13:51:53.280309 3488 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6f1b3118c44c47c0a904409826061aad [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 13:51:53.280341 3488 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6f1b3118c44c47c0a904409826061aad [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 13:51:53.280390 3488 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6f1b3118c44c47c0a904409826061aad [term 0 FOLLOWER]: Advancing to term 1
I20260924 13:51:53.281064 3488 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6f1b3118c44c47c0a904409826061aad [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f1b3118c44c47c0a904409826061aad" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 36913 } }
I20260924 13:51:53.281193 3488 leader_election.cc:302] T 00000000000000000000000000000000 P 6f1b3118c44c47c0a904409826061aad [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6f1b3118c44c47c0a904409826061aad; no voters:
I20260924 13:51:53.281373 3488 leader_election.cc:288] T 00000000000000000000000000000000 P 6f1b3118c44c47c0a904409826061aad [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 13:51:53.281423 3493 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6f1b3118c44c47c0a904409826061aad [term 1 FOLLOWER]: Leader election won for term 1
I20260924 13:51:53.281616 3493 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6f1b3118c44c47c0a904409826061aad [term 1 LEADER]: Becoming Leader. State: Replica: 6f1b3118c44c47c0a904409826061aad, State: Running, Role: LEADER
I20260924 13:51:53.281632 3488 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6f1b3118c44c47c0a904409826061aad [sys.catalog]: configured and running, proceeding with master startup.
I20260924 13:51:53.281790 3493 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6f1b3118c44c47c0a904409826061aad [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f1b3118c44c47c0a904409826061aad" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 36913 } }
I20260924 13:51:53.282159 3494 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6f1b3118c44c47c0a904409826061aad [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6f1b3118c44c47c0a904409826061aad" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f1b3118c44c47c0a904409826061aad" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 36913 } } }
I20260924 13:51:53.282243 3494 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6f1b3118c44c47c0a904409826061aad [sys.catalog]: This master's current role is: LEADER
I20260924 13:51:53.282438 3495 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6f1b3118c44c47c0a904409826061aad [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6f1b3118c44c47c0a904409826061aad. Latest consensus state: current_term: 1 leader_uuid: "6f1b3118c44c47c0a904409826061aad" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f1b3118c44c47c0a904409826061aad" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 36913 } } }
I20260924 13:51:53.282560 3495 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6f1b3118c44c47c0a904409826061aad [sys.catalog]: This master's current role is: LEADER
I20260924 13:51:53.283030 3502 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 13:51:53.283499 3502 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Sep 24 13:51:53 dist-test-slave-sjl2 krb5kdc[3407](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257913, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.149.254@KRBTEST.COM
I20260924 13:51:53.285759 3502 catalog_manager.cc:1347] Generated new cluster ID: 81fa4460773c4e05a242e214294da754
I20260924 13:51:53.285813 3502 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 13:51:53.288395 3492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:53.277002 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41783 (local address 127.13.149.254:36913)
0924 13:51:53.277177 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0924 13:51:53.277189 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0924 13:51:53.277223 (+ 34us) server_negotiation.cc:407] Connection header received
0924 13:51:53.277754 (+ 531us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:51:53.277763 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:51:53.277831 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:51:53.277988 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:51:53.278573 (+ 585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:51:53.279987 (+ 1414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:51:53.280848 (+ 861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:51:53.281571 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:51:53.282038 (+ 467us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:51:53.284690 (+ 2652us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:51:53.284695 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:51:53.284696 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 13:51:53.284710 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:51:53.286678 (+ 1968us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:51:53.287113 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:51:53.287117 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:51:53.287117 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:51:53.287218 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:51:53.287422 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:51:53.287423 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:51:53.287424 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:51:53.287707 (+ 283us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:51:53.287886 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0924 13:51:53.288003 (+ 117us) server_negotiation.cc:299] Negotiation successful
0924 13:51:53.288159 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":120,"thread_start_us":72,"threads_started":1}
I20260924 13:51:53.289372 13911 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260924 13:51:53.297497 3492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:53.290864 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48381 (local address 127.13.149.254:36913)
0924 13:51:53.290903 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0924 13:51:53.290905 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:51:53.291059 (+ 154us) server_negotiation.cc:407] Connection header received
0924 13:51:53.291113 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:51:53.291114 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:51:53.291125 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:51:53.291170 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:51:53.291704 (+ 534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:51:53.292207 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:51:53.292900 (+ 693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:51:53.293248 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:51:53.293311 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:51:53.295030 (+ 1719us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:51:53.295032 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:51:53.295033 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 13:51:53.295042 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:51:53.296378 (+ 1336us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:51:53.296734 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:51:53.296735 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:51:53.296736 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:51:53.296787 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:51:53.296980 (+ 193us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:51:53.296981 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:51:53.296981 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:51:53.297099 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:51:53.297186 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0924 13:51:53.297322 (+ 136us) server_negotiation.cc:299] Negotiation successful
0924 13:51:53.297407 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260924 13:51:53.302023 3502 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 13:51:53.302636 3502 catalog_manager.cc:1504] Loading token signing keys...
I20260924 13:51:53.304872 3502 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6f1b3118c44c47c0a904409826061aad: Generated new TSK 0
I20260924 13:51:53.305172 3502 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260924 13:51:53.308467 13911 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJN9WRF/build/release/bin/kudu with pid 3420
2026-09-24T13:51:53Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1252 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790257629041645-13911-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 13:51:53 dist-test-slave-sjl2 krb5kdc[3519](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 13:51:53 dist-test-slave-sjl2 krb5kdc[3519](info): set up 2 sockets
Sep 24 13:51:53 dist-test-slave-sjl2 krb5kdc[3519](info): commencing operation
krb5kdc: starting...
W20260924 13:51:54.266110 13911 mini_kdc.cc:121] Time spent starting KDC: real 0.940s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790257629041645-13911-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790257629041645-13911-0/krb5kdc/test-user.keytab.
Sep 24 13:51:54 dist-test-slave-sjl2 krb5kdc[3519](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257914, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T13:51:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T13:51:54Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.149.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.149.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.149.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.254.keytab.
Entry for principal kudu/127.13.149.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.254.keytab.
I20260924 13:51:54.391479 13911 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJN9WRF/build/release/bin/kudu
/tmp/dist-test-taskJN9WRF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790257629041645-13911-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790257629041645-13911-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790257629041645-13911-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790257629041645-13911-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.149.254:43097
--webserver_interface=127.13.149.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.254.keytab
--principal=kudu/127.13.149.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.149.212:42661
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.149.254:43097
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790257629041645-13911-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790257629041645-13911-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790257629041645-13911-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 13:51:54.471338 3532 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 13:51:54.471566 3532 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 13:51:54.471590 3532 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 13:51:54.472820 3532 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 13:51:54.472870 3532 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 13:51:54.472883 3532 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 13:51:54.472895 3532 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 13:51:54.472906 3532 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 13:51:54.474500 3532 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.149.212:42661
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790257629041645-13911-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790257629041645-13911-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.149.254:43097
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.254.keytab
--principal=kudu/127.13.149.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.149.254:43097
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790257629041645-13911-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.149.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3532
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790257629041645-13911-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 12:29:37 UTC on ccycloud.quasar-iqzbxn.root.comops.site
build id 4670106
I20260924 13:51:54.474747 3532 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 13:51:54.474970 3532 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260924 13:51:54.479799 3532 server_base.cc:1034] running on GCE node
W20260924 13:51:54.479770 3538 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 13:51:54.479790 3537 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 13:51:54.479770 3540 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 13:51:54.480134 3532 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 13:51:54.480443 3532 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 13:51:54.481577 3532 hybrid_clock.cc:648] HybridClock initialized: now 1790257914481557 us; error 33 us; skew 500 ppm
Sep 24 13:51:54 dist-test-slave-sjl2 krb5kdc[3519](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257914, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.149.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 13:51:54.483976 3532 init.cc:377] Logged in from keytab as kudu/127.13.149.254@KRBTEST.COM (short username kudu)
I20260924 13:51:54.484434 3532 webserver.cc:460] Webserver started at http://127.13.149.254:38303/ using document root <none> and password file <none>
I20260924 13:51:54.484650 3532 fs_manager.cc:359] Metadata directory not provided
I20260924 13:51:54.484697 3532 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 13:51:54.484800 3532 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 13:51:54.485859 3532 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790257629041645-13911-0/minicluster-data/master-0/data/instance:
uuid: "f6ca20801dc34b45ab0384bc5ff2d478"
format_stamp: "Formatted at 2026-09-24 13:51:54 on dist-test-slave-sjl2"
server_key: "94f8424618a2aa6524673c61c06a830e"
server_key_iv: "47484635fdf38a23a3c41da63d450e6d"
server_key_version: "encryptionkey@0"
I20260924 13:51:54.486176 3532 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790257629041645-13911-0/minicluster-data/master-0/wal/instance:
uuid: "f6ca20801dc34b45ab0384bc5ff2d478"
format_stamp: "Formatted at 2026-09-24 13:51:54 on dist-test-slave-sjl2"
server_key: "94f8424618a2aa6524673c61c06a830e"
server_key_iv: "47484635fdf38a23a3c41da63d450e6d"
server_key_version: "encryptionkey@0"
I20260924 13:51:54.487411 3532 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260924 13:51:54.488088 3547 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 13:51:54.488236 3532 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 13:51:54.488305 3532 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790257629041645-13911-0/minicluster-data/master-0/data,/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790257629041645-13911-0/minicluster-data/master-0/wal
uuid: "f6ca20801dc34b45ab0384bc5ff2d478"
format_stamp: "Formatted at 2026-09-24 13:51:54 on dist-test-slave-sjl2"
server_key: "94f8424618a2aa6524673c61c06a830e"
server_key_iv: "47484635fdf38a23a3c41da63d450e6d"
server_key_version: "encryptionkey@0"
I20260924 13:51:54.488371 3532 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790257629041645-13911-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790257629041645-13911-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790257629041645-13911-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 13:51:54.512432 3532 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 13:51:54.514555 3532 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 13:51:54.514671 3532 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 13:51:54.518519 3532 rpc_server.cc:307] RPC server started. Bound to: 127.13.149.254:43097
I20260924 13:51:54.518558 3599 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.149.254:43097 every 8 connection(s)
I20260924 13:51:54.518880 3532 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790257629041645-13911-0/minicluster-data/master-0/data/info.pb
I20260924 13:51:54.519718 3600 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 13:51:54.522096 3600 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f6ca20801dc34b45ab0384bc5ff2d478: Bootstrap starting.
I20260924 13:51:54.522626 3600 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f6ca20801dc34b45ab0384bc5ff2d478: Neither blocks nor log segments found. Creating new log.
I20260924 13:51:54.522863 3600 log.cc:824] T 00000000000000000000000000000000 P f6ca20801dc34b45ab0384bc5ff2d478: Log is configured to *not* fsync() on all Append() calls
I20260924 13:51:54.523553 3600 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f6ca20801dc34b45ab0384bc5ff2d478: No bootstrap required, opened a new log
I20260924 13:51:54.525189 3600 raft_consensus.cc:348] T 00000000000000000000000000000000 P f6ca20801dc34b45ab0384bc5ff2d478 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6ca20801dc34b45ab0384bc5ff2d478" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 43097 } }
I20260924 13:51:54.525339 3600 raft_consensus.cc:374] T 00000000000000000000000000000000 P f6ca20801dc34b45ab0384bc5ff2d478 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 13:51:54.525362 3600 raft_consensus.cc:729] T 00000000000000000000000000000000 P f6ca20801dc34b45ab0384bc5ff2d478 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6ca20801dc34b45ab0384bc5ff2d478, State: Initialized, Role: FOLLOWER
I20260924 13:51:54.525467 3600 consensus_queue.cc:260] T 00000000000000000000000000000000 P f6ca20801dc34b45ab0384bc5ff2d478 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6ca20801dc34b45ab0384bc5ff2d478" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 43097 } }
I20260924 13:51:54.525517 3600 raft_consensus.cc:388] T 00000000000000000000000000000000 P f6ca20801dc34b45ab0384bc5ff2d478 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 13:51:54.525542 3600 raft_consensus.cc:482] T 00000000000000000000000000000000 P f6ca20801dc34b45ab0384bc5ff2d478 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 13:51:54.525570 3600 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f6ca20801dc34b45ab0384bc5ff2d478 [term 0 FOLLOWER]: Advancing to term 1
I20260924 13:51:54.526378 3600 raft_consensus.cc:504] T 00000000000000000000000000000000 P f6ca20801dc34b45ab0384bc5ff2d478 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6ca20801dc34b45ab0384bc5ff2d478" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 43097 } }
I20260924 13:51:54.526465 3600 leader_election.cc:302] T 00000000000000000000000000000000 P f6ca20801dc34b45ab0384bc5ff2d478 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f6ca20801dc34b45ab0384bc5ff2d478; no voters:
I20260924 13:51:54.526610 3600 leader_election.cc:288] T 00000000000000000000000000000000 P f6ca20801dc34b45ab0384bc5ff2d478 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 13:51:54.526656 3603 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f6ca20801dc34b45ab0384bc5ff2d478 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 13:51:54.526690 13911 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJN9WRF/build/release/bin/kudu as pid 3532
I20260924 13:51:54.526794 13911 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790257629041645-13911-0/minicluster-data/master-0/wal/instance
I20260924 13:51:54.526821 3603 raft_consensus.cc:686] T 00000000000000000000000000000000 P f6ca20801dc34b45ab0384bc5ff2d478 [term 1 LEADER]: Becoming Leader. State: Replica: f6ca20801dc34b45ab0384bc5ff2d478, State: Running, Role: LEADER
I20260924 13:51:54.526870 3600 sys_catalog.cc:564] T 00000000000000000000000000000000 P f6ca20801dc34b45ab0384bc5ff2d478 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 13:51:54.526964 13911 external_mini_cluster.cc:1428] Setting key bed2686c3288804f0e4d164bea40a924
I20260924 13:51:54.526999 3603 consensus_queue.cc:237] T 00000000000000000000000000000000 P f6ca20801dc34b45ab0384bc5ff2d478 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6ca20801dc34b45ab0384bc5ff2d478" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 43097 } }
I20260924 13:51:54.527455 3603 sys_catalog.cc:455] T 00000000000000000000000000000000 P f6ca20801dc34b45ab0384bc5ff2d478 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f6ca20801dc34b45ab0384bc5ff2d478. Latest consensus state: current_term: 1 leader_uuid: "f6ca20801dc34b45ab0384bc5ff2d478" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6ca20801dc34b45ab0384bc5ff2d478" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 43097 } } }
I20260924 13:51:54.527510 3605 sys_catalog.cc:455] T 00000000000000000000000000000000 P f6ca20801dc34b45ab0384bc5ff2d478 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f6ca20801dc34b45ab0384bc5ff2d478" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6ca20801dc34b45ab0384bc5ff2d478" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 43097 } } }
I20260924 13:51:54.527606 3605 sys_catalog.cc:458] T 00000000000000000000000000000000 P f6ca20801dc34b45ab0384bc5ff2d478 [sys.catalog]: This master's current role is: LEADER
I20260924 13:51:54.527767 3603 sys_catalog.cc:458] T 00000000000000000000000000000000 P f6ca20801dc34b45ab0384bc5ff2d478 [sys.catalog]: This master's current role is: LEADER
I20260924 13:51:54.527885 3614 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 13:51:54.528452 3614 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 13:51:54.531111 3614 catalog_manager.cc:1347] Generated new cluster ID: fc8db520553f4957b5d7817d7cf1727f
I20260924 13:51:54.531168 3614 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 13:51:54.538004 3614 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 13:51:54.538460 3610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:54.527434 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56443 (local address 127.13.149.254:43097)
0924 13:51:54.527802 (+ 368us) server_negotiation.cc:206] Beginning negotiation
0924 13:51:54.527811 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0924 13:51:54.527845 (+ 34us) server_negotiation.cc:407] Connection header received
0924 13:51:54.527970 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:51:54.527978 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:51:54.528042 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:51:54.528196 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:51:54.528431 (+ 235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:51:54.529976 (+ 1545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:51:54.530872 (+ 896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:51:54.531473 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:51:54.531586 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:51:54.537535 (+ 5949us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:51:54.537541 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:51:54.537541 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 13:51:54.537771 (+ 230us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:51:54.537820 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 13:51:54.537839 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:51:54.537921 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0924 13:51:54.538140 (+ 219us) server_negotiation.cc:299] Negotiation successful
0924 13:51:54.538228 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":315,"thread_start_us":62,"threads_started":1}
I20260924 13:51:54.539001 3614 catalog_manager.cc:1504] Loading token signing keys...
I20260924 13:51:54.539407 13911 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260924 13:51:54.543474 3610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:54.540720 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48577 (local address 127.13.149.254:43097)
0924 13:51:54.540757 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0924 13:51:54.540758 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0924 13:51:54.540866 (+ 108us) server_negotiation.cc:407] Connection header received
0924 13:51:54.540941 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:51:54.540942 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:51:54.540953 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:51:54.541000 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:51:54.541120 (+ 120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:51:54.541608 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:51:54.542331 (+ 723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:51:54.542552 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:51:54.542606 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:51:54.543244 (+ 638us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:51:54.543245 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:51:54.543245 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 13:51:54.543252 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:51:54.543267 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 13:51:54.543272 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:51:54.543314 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0924 13:51:54.543364 (+ 50us) server_negotiation.cc:299] Negotiation successful
0924 13:51:54.543395 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260924 13:51:54.548571 3614 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f6ca20801dc34b45ab0384bc5ff2d478: Generated new TSK 0
I20260924 13:51:54.548779 3614 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260924 13:51:54.553907 13911 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJN9WRF/build/release/bin/kudu with pid 3532
2026-09-24T13:51:54Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1245 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790257629041645-13911-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 13:51:54 dist-test-slave-sjl2 krb5kdc[3631](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 13:51:54 dist-test-slave-sjl2 krb5kdc[3631](info): set up 2 sockets
Sep 24 13:51:54 dist-test-slave-sjl2 krb5kdc[3631](info): commencing operation
krb5kdc: starting...
W20260924 13:51:55.506052 13911 mini_kdc.cc:121] Time spent starting KDC: real 0.935s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790257629041645-13911-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790257629041645-13911-0/krb5kdc/test-user.keytab.
Sep 24 13:51:55 dist-test-slave-sjl2 krb5kdc[3631](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257915, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T13:51:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T13:51:55Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.149.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.149.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.149.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.254.keytab.
Entry for principal kudu/127.13.149.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.254.keytab.
I20260924 13:51:55.624198 13911 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJN9WRF/build/release/bin/kudu
/tmp/dist-test-taskJN9WRF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790257629041645-13911-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790257629041645-13911-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790257629041645-13911-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790257629041645-13911-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.149.254:37811
--webserver_interface=127.13.149.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.254.keytab
--principal=kudu/127.13.149.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.149.212:42811
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.149.254:37811
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790257629041645-13911-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790257629041645-13911-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790257629041645-13911-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 13:51:55.703053 3644 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 13:51:55.703266 3644 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 13:51:55.703289 3644 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 13:51:55.704454 3644 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 13:51:55.704506 3644 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 13:51:55.704519 3644 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 13:51:55.704530 3644 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 13:51:55.704542 3644 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 13:51:55.706091 3644 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.149.212:42811
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790257629041645-13911-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790257629041645-13911-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.149.254:37811
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.254.keytab
--principal=kudu/127.13.149.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.149.254:37811
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790257629041645-13911-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.149.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3644
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790257629041645-13911-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 12:29:37 UTC on ccycloud.quasar-iqzbxn.root.comops.site
build id 4670106
I20260924 13:51:55.706329 3644 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 13:51:55.706555 3644 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 13:51:55.711261 3650 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 13:51:55.711272 3652 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 13:51:55.711272 3649 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 13:51:55.711302 3644 server_base.cc:1034] running on GCE node
I20260924 13:51:55.711771 3644 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 13:51:55.712033 3644 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 13:51:55.713207 3644 hybrid_clock.cc:648] HybridClock initialized: now 1790257915713190 us; error 32 us; skew 500 ppm
Sep 24 13:51:55 dist-test-slave-sjl2 krb5kdc[3631](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257915, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.149.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 13:51:55.715464 3644 init.cc:377] Logged in from keytab as kudu/127.13.149.254@KRBTEST.COM (short username kudu)
I20260924 13:51:55.715878 3644 webserver.cc:460] Webserver started at http://127.13.149.254:44443/ using document root <none> and password file <none>
I20260924 13:51:55.716094 3644 fs_manager.cc:359] Metadata directory not provided
I20260924 13:51:55.716142 3644 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 13:51:55.716274 3644 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 13:51:55.717414 3644 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790257629041645-13911-0/minicluster-data/master-0/data/instance:
uuid: "ae091e49fb384fab9f7b4c68a68c2aff"
format_stamp: "Formatted at 2026-09-24 13:51:55 on dist-test-slave-sjl2"
server_key: "d8dedd7aaa767a0e094c2bae2827988e"
server_key_iv: "ca9c3869092d3a5ee0e5df91b6af5aa6"
server_key_version: "encryptionkey@0"
I20260924 13:51:55.717731 3644 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790257629041645-13911-0/minicluster-data/master-0/wal/instance:
uuid: "ae091e49fb384fab9f7b4c68a68c2aff"
format_stamp: "Formatted at 2026-09-24 13:51:55 on dist-test-slave-sjl2"
server_key: "d8dedd7aaa767a0e094c2bae2827988e"
server_key_iv: "ca9c3869092d3a5ee0e5df91b6af5aa6"
server_key_version: "encryptionkey@0"
I20260924 13:51:55.718935 3644 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260924 13:51:55.719589 3659 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 13:51:55.719767 3644 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 13:51:55.719851 3644 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790257629041645-13911-0/minicluster-data/master-0/data,/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790257629041645-13911-0/minicluster-data/master-0/wal
uuid: "ae091e49fb384fab9f7b4c68a68c2aff"
format_stamp: "Formatted at 2026-09-24 13:51:55 on dist-test-slave-sjl2"
server_key: "d8dedd7aaa767a0e094c2bae2827988e"
server_key_iv: "ca9c3869092d3a5ee0e5df91b6af5aa6"
server_key_version: "encryptionkey@0"
I20260924 13:51:55.719911 3644 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790257629041645-13911-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790257629041645-13911-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790257629041645-13911-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 13:51:55.723214 3644 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 13:51:55.725219 3644 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 13:51:55.725351 3644 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 13:51:55.729094 3644 rpc_server.cc:307] RPC server started. Bound to: 127.13.149.254:37811
I20260924 13:51:55.729122 3711 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.149.254:37811 every 8 connection(s)
I20260924 13:51:55.729455 3644 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790257629041645-13911-0/minicluster-data/master-0/data/info.pb
I20260924 13:51:55.730396 3712 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 13:51:55.732647 3712 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ae091e49fb384fab9f7b4c68a68c2aff: Bootstrap starting.
I20260924 13:51:55.733258 3712 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ae091e49fb384fab9f7b4c68a68c2aff: Neither blocks nor log segments found. Creating new log.
I20260924 13:51:55.733515 3712 log.cc:824] T 00000000000000000000000000000000 P ae091e49fb384fab9f7b4c68a68c2aff: Log is configured to *not* fsync() on all Append() calls
I20260924 13:51:55.734174 3712 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ae091e49fb384fab9f7b4c68a68c2aff: No bootstrap required, opened a new log
I20260924 13:51:55.735719 3712 raft_consensus.cc:348] T 00000000000000000000000000000000 P ae091e49fb384fab9f7b4c68a68c2aff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae091e49fb384fab9f7b4c68a68c2aff" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 37811 } }
I20260924 13:51:55.735877 3712 raft_consensus.cc:374] T 00000000000000000000000000000000 P ae091e49fb384fab9f7b4c68a68c2aff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 13:51:55.735904 3712 raft_consensus.cc:729] T 00000000000000000000000000000000 P ae091e49fb384fab9f7b4c68a68c2aff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ae091e49fb384fab9f7b4c68a68c2aff, State: Initialized, Role: FOLLOWER
I20260924 13:51:55.736032 3712 consensus_queue.cc:260] T 00000000000000000000000000000000 P ae091e49fb384fab9f7b4c68a68c2aff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae091e49fb384fab9f7b4c68a68c2aff" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 37811 } }
I20260924 13:51:55.736090 3712 raft_consensus.cc:388] T 00000000000000000000000000000000 P ae091e49fb384fab9f7b4c68a68c2aff [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 13:51:55.736119 3712 raft_consensus.cc:482] T 00000000000000000000000000000000 P ae091e49fb384fab9f7b4c68a68c2aff [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 13:51:55.736166 3712 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ae091e49fb384fab9f7b4c68a68c2aff [term 0 FOLLOWER]: Advancing to term 1
I20260924 13:51:55.737030 3712 raft_consensus.cc:504] T 00000000000000000000000000000000 P ae091e49fb384fab9f7b4c68a68c2aff [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae091e49fb384fab9f7b4c68a68c2aff" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 37811 } }
I20260924 13:51:55.737123 3712 leader_election.cc:302] T 00000000000000000000000000000000 P ae091e49fb384fab9f7b4c68a68c2aff [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ae091e49fb384fab9f7b4c68a68c2aff; no voters:
I20260924 13:51:55.737293 3712 leader_election.cc:288] T 00000000000000000000000000000000 P ae091e49fb384fab9f7b4c68a68c2aff [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 13:51:55.737337 3715 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ae091e49fb384fab9f7b4c68a68c2aff [term 1 FOLLOWER]: Leader election won for term 1
I20260924 13:51:55.737490 3715 raft_consensus.cc:686] T 00000000000000000000000000000000 P ae091e49fb384fab9f7b4c68a68c2aff [term 1 LEADER]: Becoming Leader. State: Replica: ae091e49fb384fab9f7b4c68a68c2aff, State: Running, Role: LEADER
I20260924 13:51:55.737536 3712 sys_catalog.cc:564] T 00000000000000000000000000000000 P ae091e49fb384fab9f7b4c68a68c2aff [sys.catalog]: configured and running, proceeding with master startup.
I20260924 13:51:55.737578 3715 consensus_queue.cc:237] T 00000000000000000000000000000000 P ae091e49fb384fab9f7b4c68a68c2aff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae091e49fb384fab9f7b4c68a68c2aff" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 37811 } }
I20260924 13:51:55.737936 3716 sys_catalog.cc:455] T 00000000000000000000000000000000 P ae091e49fb384fab9f7b4c68a68c2aff [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ae091e49fb384fab9f7b4c68a68c2aff" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae091e49fb384fab9f7b4c68a68c2aff" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 37811 } } }
I20260924 13:51:55.737960 3717 sys_catalog.cc:455] T 00000000000000000000000000000000 P ae091e49fb384fab9f7b4c68a68c2aff [sys.catalog]: SysCatalogTable state changed. Reason: New leader ae091e49fb384fab9f7b4c68a68c2aff. Latest consensus state: current_term: 1 leader_uuid: "ae091e49fb384fab9f7b4c68a68c2aff" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae091e49fb384fab9f7b4c68a68c2aff" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 37811 } } }
I20260924 13:51:55.738088 3717 sys_catalog.cc:458] T 00000000000000000000000000000000 P ae091e49fb384fab9f7b4c68a68c2aff [sys.catalog]: This master's current role is: LEADER
I20260924 13:51:55.738287 3724 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 13:51:55.738459 3716 sys_catalog.cc:458] T 00000000000000000000000000000000 P ae091e49fb384fab9f7b4c68a68c2aff [sys.catalog]: This master's current role is: LEADER
I20260924 13:51:55.738601 13911 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJN9WRF/build/release/bin/kudu as pid 3644
I20260924 13:51:55.738706 13911 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790257629041645-13911-0/minicluster-data/master-0/wal/instance
I20260924 13:51:55.738797 3724 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 13:51:55.738884 13911 external_mini_cluster.cc:1428] Setting key f2f4f750805c502423660184020db2a4
I20260924 13:51:55.740782 3724 catalog_manager.cc:1347] Generated new cluster ID: c165a4fe12ea42ee91c9b8e68d48c9eb
I20260924 13:51:55.740834 3724 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 13:51:55.741217 3734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:55.739497 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59381 (local address 127.13.149.254:37811)
0924 13:51:55.739824 (+ 327us) server_negotiation.cc:206] Beginning negotiation
0924 13:51:55.739833 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0924 13:51:55.739865 (+ 32us) server_negotiation.cc:407] Connection header received
0924 13:51:55.739978 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:51:55.739985 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:51:55.740014 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:51:55.740179 (+ 165us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:51:55.740765 (+ 586us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:51:55.740769 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:51:55.740769 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 13:51:55.740774 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:51:55.740822 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 13:51:55.740836 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:51:55.740874 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0924 13:51:55.741012 (+ 138us) server_negotiation.cc:299] Negotiation successful
0924 13:51:55.741057 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":284,"thread_start_us":63,"threads_started":1}
I20260924 13:51:55.742080 13911 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260924 13:51:55.749756 3724 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 13:51:55.749931 3724 catalog_manager.cc:1504] Loading token signing keys...
I20260924 13:51:55.750538 3734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:55.749376 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60221 (local address 127.13.149.254:37811)
0924 13:51:55.749416 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0924 13:51:55.749418 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:51:55.749576 (+ 158us) server_negotiation.cc:407] Connection header received
0924 13:51:55.749666 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:51:55.749667 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:51:55.749674 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:51:55.749736 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:51:55.750334 (+ 598us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:51:55.750336 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:51:55.750336 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 13:51:55.750341 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:51:55.750357 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 13:51:55.750361 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:51:55.750406 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0924 13:51:55.750456 (+ 50us) server_negotiation.cc:299] Negotiation successful
0924 13:51:55.750469 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260924 13:51:55.752743 3724 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ae091e49fb384fab9f7b4c68a68c2aff: Generated new TSK 0
I20260924 13:51:55.752907 3724 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260924 13:51:55.758296 13911 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJN9WRF/build/release/bin/kudu with pid 3644
2026-09-24T13:51:55Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1204 ms)
[----------] 18 tests from AuthTokenIssuingTest (22176 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790257629041645-13911-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 13:51:55 dist-test-slave-sjl2 krb5kdc[3743](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 13:51:55 dist-test-slave-sjl2 krb5kdc[3743](info): set up 2 sockets
Sep 24 13:51:55 dist-test-slave-sjl2 krb5kdc[3743](info): commencing operation
krb5kdc: starting...
W20260924 13:51:56.714138 13911 mini_kdc.cc:121] Time spent starting KDC: real 0.939s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790257629041645-13911-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790257629041645-13911-0/krb5kdc/test-user.keytab.
Sep 24 13:51:56 dist-test-slave-sjl2 krb5kdc[3743](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257916, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T13:51:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T13:51:56Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.149.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.149.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.149.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.254.keytab.
Entry for principal kudu/127.13.149.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.254.keytab.
I20260924 13:51:56.832538 13911 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJN9WRF/build/release/bin/kudu
/tmp/dist-test-taskJN9WRF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790257629041645-13911-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790257629041645-13911-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790257629041645-13911-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790257629041645-13911-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.149.254:40929
--webserver_interface=127.13.149.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.254.keytab
--principal=kudu/127.13.149.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.149.212:36885
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.149.254:40929,127.13.149.253:36629,127.13.149.252:37231
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790257629041645-13911-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790257629041645-13911-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790257629041645-13911-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 13:51:56.912492 3756 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260924 13:51:56.912734 3756 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 13:51:56.912750 3756 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 13:51:56.912770 3756 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 13:51:56.914062 3756 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 13:51:56.914127 3756 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 13:51:56.914140 3756 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 13:51:56.914152 3756 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 13:51:56.914163 3756 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 13:51:56.915823 3756 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.149.212:36885
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790257629041645-13911-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790257629041645-13911-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.149.254:40929,127.13.149.253:36629,127.13.149.252:37231
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.254.keytab
--principal=kudu/127.13.149.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.149.254:40929
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790257629041645-13911-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.149.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3756
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790257629041645-13911-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 12:29:37 UTC on ccycloud.quasar-iqzbxn.root.comops.site
build id 4670106
I20260924 13:51:56.916100 3756 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 13:51:56.916342 3756 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 13:51:56.921208 3761 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 13:51:56.921208 3762 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 13:51:56.921273 3764 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 13:51:56.921715 3756 server_base.cc:1034] running on GCE node
I20260924 13:51:56.921979 3756 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 13:51:56.922297 3756 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 13:51:56.923451 3756 hybrid_clock.cc:648] HybridClock initialized: now 1790257916923434 us; error 40 us; skew 500 ppm
Sep 24 13:51:56 dist-test-slave-sjl2 krb5kdc[3743](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257916, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.149.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 13:51:56.925925 3756 init.cc:377] Logged in from keytab as kudu/127.13.149.254@KRBTEST.COM (short username kudu)
I20260924 13:51:56.926399 3756 webserver.cc:460] Webserver started at http://127.13.149.254:39903/ using document root <none> and password file <none>
I20260924 13:51:56.926596 3756 fs_manager.cc:359] Metadata directory not provided
I20260924 13:51:56.926643 3756 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 13:51:56.926767 3756 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 13:51:56.927862 3756 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790257629041645-13911-0/minicluster-data/master-0/data/instance:
uuid: "252a98bd316f447c8b2f98ea1802a935"
format_stamp: "Formatted at 2026-09-24 13:51:56 on dist-test-slave-sjl2"
server_key: "aecf3e859e1be16861839cb332c12351"
server_key_iv: "c713366047cf297cb7b0597eb7ddad9c"
server_key_version: "encryptionkey@0"
I20260924 13:51:56.928205 3756 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790257629041645-13911-0/minicluster-data/master-0/wal/instance:
uuid: "252a98bd316f447c8b2f98ea1802a935"
format_stamp: "Formatted at 2026-09-24 13:51:56 on dist-test-slave-sjl2"
server_key: "aecf3e859e1be16861839cb332c12351"
server_key_iv: "c713366047cf297cb7b0597eb7ddad9c"
server_key_version: "encryptionkey@0"
I20260924 13:51:56.929508 3756 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260924 13:51:56.930225 3771 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 13:51:56.930424 3756 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260924 13:51:56.930509 3756 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790257629041645-13911-0/minicluster-data/master-0/data,/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790257629041645-13911-0/minicluster-data/master-0/wal
uuid: "252a98bd316f447c8b2f98ea1802a935"
format_stamp: "Formatted at 2026-09-24 13:51:56 on dist-test-slave-sjl2"
server_key: "aecf3e859e1be16861839cb332c12351"
server_key_iv: "c713366047cf297cb7b0597eb7ddad9c"
server_key_version: "encryptionkey@0"
I20260924 13:51:56.930569 3756 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790257629041645-13911-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790257629041645-13911-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790257629041645-13911-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 13:51:56.944209 3756 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 13:51:56.946202 3756 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 13:51:56.946345 3756 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 13:51:56.950374 3756 rpc_server.cc:307] RPC server started. Bound to: 127.13.149.254:40929
I20260924 13:51:56.950413 3823 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.149.254:40929 every 8 connection(s)
I20260924 13:51:56.950752 3756 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790257629041645-13911-0/minicluster-data/master-0/data/info.pb
I20260924 13:51:56.951594 3824 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 13:51:56.953191 3824 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.149.254" port: 40929 } has no permanent_uuid. Determining permanent_uuid...
I20260924 13:51:56.957540 13911 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJN9WRF/build/release/bin/kudu as pid 3756
I20260924 13:51:56.957666 13911 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790257629041645-13911-0/minicluster-data/master-0/wal/instance
I20260924 13:51:56.957867 13911 external_mini_cluster.cc:1428] Setting key 84e514afb431cb424ba9b69918eb097b
Sep 24 13:51:56 dist-test-slave-sjl2 krb5kdc[3743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257916, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.149.254@KRBTEST.COM for kudu/127.13.149.254@KRBTEST.COM
I20260924 13:51:56.963424 3827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:56.953914 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45760 (local address 127.13.149.254:40929)
0924 13:51:56.954127 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0924 13:51:56.954131 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 13:51:56.954185 (+ 54us) server_negotiation.cc:407] Connection header received
0924 13:51:56.954719 (+ 534us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:51:56.954720 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:51:56.954738 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:51:56.954815 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:51:56.955416 (+ 601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:51:56.956221 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:51:56.957146 (+ 925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:51:56.957462 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:51:56.957538 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:51:56.959908 (+ 2370us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:51:56.959917 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:51:56.959917 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 13:51:56.959927 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:51:56.961530 (+ 1603us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:51:56.961981 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:51:56.961984 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:51:56.961984 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:51:56.962069 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:51:56.962424 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:51:56.962425 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:51:56.962425 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:51:56.962654 (+ 229us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:51:56.962741 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0924 13:51:56.963026 (+ 285us) server_negotiation.cc:299] Negotiation successful
0924 13:51:56.963228 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":99,"server-negotiator.queue_time_us":190,"thread_start_us":78,"threads_started":1}
I20260924 13:51:56.963438 3826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:56.953893 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:40929 (local address 127.0.0.1:45760)
0924 13:51:56.954066 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0924 13:51:56.954102 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0924 13:51:56.954581 (+ 479us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:51:56.954878 (+ 297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:51:56.954883 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:51:56.955170 (+ 287us) client_negotiation.cc:189] Negotiated authn=SASL
0924 13:51:56.955334 (+ 164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:51:56.955338 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:51:56.956314 (+ 976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:51:56.956315 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:51:56.957076 (+ 761us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:51:56.957079 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:51:56.957548 (+ 469us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:51:56.957549 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:51:56.957662 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:51:56.958216 (+ 554us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 13:51:56.958225 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0924 13:51:56.959809 (+ 1584us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 13:51:56.961614 (+ 1805us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 13:51:56.961621 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 13:51:56.961624 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0924 13:51:56.961886 (+ 262us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 13:51:56.962204 (+ 318us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 13:51:56.962205 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 13:51:56.962205 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 13:51:56.962318 (+ 113us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 13:51:56.962739 (+ 421us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 13:51:56.962746 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 13:51:56.962827 (+ 81us) client_negotiation.cc:769] Sending connection context
0924 13:51:56.962947 (+ 120us) client_negotiation.cc:240] Negotiation successful
0924 13:51:56.963137 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":81,"threads_started":1}
I20260924 13:51:56.964268 3824 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.149.253" port: 36629 } has no permanent_uuid. Determining permanent_uuid...
I20260924 13:51:56.964584 3826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:56.964468 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.253:36629 (local address 127.0.0.1:44662)
0924 13:51:56.964506 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0924 13:51:56.964556 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.149.253:36629: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":18}
W20260924 13:51:56.964716 3773 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.13.149.253:36629: connect: Connection refused (error 111)
I20260924 13:51:56.965137 3826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:56.965030 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.253:36629 (local address 127.0.0.1:44678)
0924 13:51:56.965081 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0924 13:51:56.965112 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.149.253:36629: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260924 13:51:56.965314 3824 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.149.253:36629: Network error: Client connection negotiation failed: client connection to 127.13.149.253:36629: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.13.149.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.149.253@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.149.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.253.keytab.
Entry for principal kudu/127.13.149.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.253.keytab.
I20260924 13:51:56.984529 13911 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJN9WRF/build/release/bin/kudu
/tmp/dist-test-taskJN9WRF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790257629041645-13911-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790257629041645-13911-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790257629041645-13911-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790257629041645-13911-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.149.253:36629
--webserver_interface=127.13.149.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.253.keytab
--principal=kudu/127.13.149.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.149.212:36885
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.149.254:40929,127.13.149.253:36629,127.13.149.252:37231
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790257629041645-13911-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790257629041645-13911-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790257629041645-13911-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260924 13:51:57.014559 3824 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.149.253" port: 36629 } attempt: 1
I20260924 13:51:57.015127 3826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:57.014975 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.253:36629 (local address 127.0.0.1:44684)
0924 13:51:57.015040 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0924 13:51:57.015090 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.149.253:36629: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260924 13:51:57.015563 3826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:57.015472 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.253:36629 (local address 127.0.0.1:44700)
0924 13:51:57.015511 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0924 13:51:57.015539 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.149.253:36629: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
W20260924 13:51:57.015702 3824 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.149.253:36629: Network error: Client connection negotiation failed: client connection to 127.13.149.253:36629: connect: Connection refused (error 111)
W20260924 13:51:57.066090 3831 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260924 13:51:57.066313 3831 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 13:51:57.066330 3831 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 13:51:57.066350 3831 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 13:51:57.067566 3831 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 13:51:57.067612 3831 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 13:51:57.067625 3831 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 13:51:57.067636 3831 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 13:51:57.067647 3831 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 13:51:57.069209 3831 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.149.212:36885
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790257629041645-13911-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790257629041645-13911-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.13.149.254:40929,127.13.149.253:36629,127.13.149.252:37231
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.253.keytab
--principal=kudu/127.13.149.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.149.253:36629
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790257629041645-13911-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.13.149.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3831
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790257629041645-13911-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 12:29:37 UTC on ccycloud.quasar-iqzbxn.root.comops.site
build id 4670106
I20260924 13:51:57.069458 3831 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 13:51:57.069671 3831 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 13:51:57.074370 3837 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 13:51:57.074394 3839 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 13:51:57.074375 3836 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 13:51:57.074591 3831 server_base.cc:1034] running on GCE node
I20260924 13:51:57.074815 3831 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 13:51:57.075076 3831 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 13:51:57.076210 3831 hybrid_clock.cc:648] HybridClock initialized: now 1790257917076197 us; error 31 us; skew 500 ppm
Sep 24 13:51:57 dist-test-slave-sjl2 krb5kdc[3743](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257917, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.149.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 13:51:57.078749 3831 init.cc:377] Logged in from keytab as kudu/127.13.149.253@KRBTEST.COM (short username kudu)
I20260924 13:51:57.079227 3831 webserver.cc:460] Webserver started at http://127.13.149.253:40241/ using document root <none> and password file <none>
I20260924 13:51:57.079454 3831 fs_manager.cc:359] Metadata directory not provided
I20260924 13:51:57.079499 3831 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 13:51:57.079607 3831 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 13:51:57.080660 3831 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790257629041645-13911-0/minicluster-data/master-1/data/instance:
uuid: "9c78d1aa2ebc4ea7899bcc2a1a0ef022"
format_stamp: "Formatted at 2026-09-24 13:51:57 on dist-test-slave-sjl2"
server_key: "b3fa3819fcb1e85b9e187e67f3a3c003"
server_key_iv: "97f4c925396743ba698aad1250bfdee7"
server_key_version: "encryptionkey@0"
I20260924 13:51:57.081007 3831 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790257629041645-13911-0/minicluster-data/master-1/wal/instance:
uuid: "9c78d1aa2ebc4ea7899bcc2a1a0ef022"
format_stamp: "Formatted at 2026-09-24 13:51:57 on dist-test-slave-sjl2"
server_key: "b3fa3819fcb1e85b9e187e67f3a3c003"
server_key_iv: "97f4c925396743ba698aad1250bfdee7"
server_key_version: "encryptionkey@0"
I20260924 13:51:57.082372 3831 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260924 13:51:57.083125 3846 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 13:51:57.083315 3831 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260924 13:51:57.083400 3831 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790257629041645-13911-0/minicluster-data/master-1/data,/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790257629041645-13911-0/minicluster-data/master-1/wal
uuid: "9c78d1aa2ebc4ea7899bcc2a1a0ef022"
format_stamp: "Formatted at 2026-09-24 13:51:57 on dist-test-slave-sjl2"
server_key: "b3fa3819fcb1e85b9e187e67f3a3c003"
server_key_iv: "97f4c925396743ba698aad1250bfdee7"
server_key_version: "encryptionkey@0"
I20260924 13:51:57.083463 3831 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790257629041645-13911-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790257629041645-13911-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790257629041645-13911-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 13:51:57.083850 3824 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.149.253" port: 36629 } attempt: 2
I20260924 13:51:57.084242 3826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:57.084097 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.253:36629 (local address 127.0.0.1:44702)
0924 13:51:57.084181 (+ 84us) negotiation.cc:107] Waiting for socket to connect
0924 13:51:57.084216 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.149.253:36629: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260924 13:51:57.084524 3826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:57.084452 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.253:36629 (local address 127.0.0.1:44712)
0924 13:51:57.084485 (+ 33us) negotiation.cc:107] Waiting for socket to connect
0924 13:51:57.084508 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.149.253:36629: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":21}
W20260924 13:51:57.084594 3824 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.149.253:36629: Network error: Client connection negotiation failed: client connection to 127.13.149.253:36629: connect: Connection refused (error 111)
I20260924 13:51:57.100687 3831 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 13:51:57.102722 3831 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 13:51:57.102883 3831 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 13:51:57.106809 3831 rpc_server.cc:307] RPC server started. Bound to: 127.13.149.253:36629
I20260924 13:51:57.106858 3898 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.149.253:36629 every 8 connection(s)
I20260924 13:51:57.107191 3831 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790257629041645-13911-0/minicluster-data/master-1/data/info.pb
I20260924 13:51:57.108103 3899 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 13:51:57.109462 13911 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJN9WRF/build/release/bin/kudu as pid 3831
I20260924 13:51:57.109566 13911 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790257629041645-13911-0/minicluster-data/master-1/wal/instance
I20260924 13:51:57.109711 13911 external_mini_cluster.cc:1428] Setting key 99d01233d69bc271b432544dd989ea29
I20260924 13:51:57.109706 3899 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.149.254" port: 40929 } has no permanent_uuid. Determining permanent_uuid...
Sep 24 13:51:57 dist-test-slave-sjl2 krb5kdc[3743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257917, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.149.253@KRBTEST.COM for kudu/127.13.149.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.13.149.252@KRBTEST.COM; defaulting to no policy
I20260924 13:51:57.118669 3827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:57.110469 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45770 (local address 127.13.149.254:40929)
0924 13:51:57.110520 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0924 13:51:57.110522 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:51:57.110722 (+ 200us) server_negotiation.cc:407] Connection header received
0924 13:51:57.111350 (+ 628us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:51:57.111352 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:51:57.111366 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:51:57.111416 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:51:57.112183 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:51:57.112674 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:51:57.113692 (+ 1018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:51:57.113886 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:51:57.113941 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:51:57.116116 (+ 2175us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:51:57.116119 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:51:57.116119 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 13:51:57.116128 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:51:57.117441 (+ 1313us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:51:57.117830 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:51:57.117831 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:51:57.117832 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:51:57.117882 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:51:57.118059 (+ 177us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:51:57.118059 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:51:57.118060 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:51:57.118174 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:51:57.118224 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0924 13:51:57.118486 (+ 262us) server_negotiation.cc:299] Negotiation successful
0924 13:51:57.118574 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260924 13:51:57.118842 3901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:57.110452 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:40929 (local address 127.0.0.1:45770)
0924 13:51:57.110599 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0924 13:51:57.110634 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0924 13:51:57.111239 (+ 605us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:51:57.111492 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:51:57.111496 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:51:57.111837 (+ 341us) client_negotiation.cc:189] Negotiated authn=SASL
0924 13:51:57.112091 (+ 254us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:51:57.112097 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:51:57.112753 (+ 656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:51:57.112754 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:51:57.113608 (+ 854us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:51:57.113610 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:51:57.113951 (+ 341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:51:57.113953 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:51:57.114082 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:51:57.114551 (+ 469us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 13:51:57.114559 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0924 13:51:57.116019 (+ 1460us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 13:51:57.117519 (+ 1500us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 13:51:57.117525 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 13:51:57.117527 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 13:51:57.117769 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 13:51:57.117932 (+ 163us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 13:51:57.117932 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 13:51:57.117933 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 13:51:57.118018 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 13:51:57.118219 (+ 201us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 13:51:57.118222 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 13:51:57.118332 (+ 110us) client_negotiation.cc:769] Sending connection context
0924 13:51:57.118442 (+ 110us) client_negotiation.cc:240] Negotiation successful
0924 13:51:57.118625 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":76,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.149.252@KRBTEST.COM" created.
I20260924 13:51:57.119575 3899 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.149.253" port: 36629 } has no permanent_uuid. Determining permanent_uuid...
Sep 24 13:51:57 dist-test-slave-sjl2 krb5kdc[3743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257917, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.149.253@KRBTEST.COM for kudu/127.13.149.253@KRBTEST.COM
I20260924 13:51:57.127209 3901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:57.119834 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.253:36629 (local address 127.0.0.1:44722)
0924 13:51:57.119878 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0924 13:51:57.119888 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 13:51:57.119928 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:51:57.120169 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:51:57.120170 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:51:57.120282 (+ 112us) client_negotiation.cc:189] Negotiated authn=SASL
0924 13:51:57.120333 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:51:57.120334 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:51:57.121306 (+ 972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:51:57.121307 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:51:57.121887 (+ 580us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:51:57.121889 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:51:57.122498 (+ 609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:51:57.122499 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:51:57.122570 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:51:57.123003 (+ 433us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 13:51:57.123008 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 13:51:57.124182 (+ 1174us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 13:51:57.126218 (+ 2036us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 13:51:57.126219 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 13:51:57.126220 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 13:51:57.126387 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 13:51:57.126614 (+ 227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 13:51:57.126614 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 13:51:57.126615 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 13:51:57.126668 (+ 53us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 13:51:57.126951 (+ 283us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 13:51:57.126951 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 13:51:57.126989 (+ 38us) client_negotiation.cc:769] Sending connection context
0924 13:51:57.127050 (+ 61us) client_negotiation.cc:240] Negotiation successful
0924 13:51:57.127135 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":27}
I20260924 13:51:57.127310 3903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:57.119871 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44722 (local address 127.13.149.253:36629)
0924 13:51:57.120028 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0924 13:51:57.120032 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 13:51:57.120051 (+ 19us) server_negotiation.cc:407] Connection header received
0924 13:51:57.120072 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:51:57.120074 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:51:57.120088 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:51:57.120167 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:51:57.120392 (+ 225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:51:57.121231 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:51:57.121963 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:51:57.122423 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:51:57.122495 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:51:57.124279 (+ 1784us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:51:57.124290 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:51:57.124290 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 13:51:57.124298 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:51:57.126118 (+ 1820us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:51:57.126477 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:51:57.126481 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:51:57.126481 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:51:57.126552 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:51:57.126726 (+ 174us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:51:57.126727 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:51:57.126727 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:51:57.126901 (+ 174us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:51:57.126964 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0924 13:51:57.127076 (+ 112us) server_negotiation.cc:299] Negotiation successful
0924 13:51:57.127189 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":42,"server-negotiator.queue_time_us":132,"thread_start_us":96,"threads_started":1}
I20260924 13:51:57.127933 3899 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.149.252" port: 37231 } has no permanent_uuid. Determining permanent_uuid...
I20260924 13:51:57.128324 3901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:57.128206 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.252:37231 (local address 127.0.0.1:50890)
0924 13:51:57.128251 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0924 13:51:57.128298 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.149.252:37231: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260924 13:51:57.128440 3848 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.13.149.252:37231: connect: Connection refused (error 111)
I20260924 13:51:57.128809 3901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:57.128720 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.252:37231 (local address 127.0.0.1:50892)
0924 13:51:57.128764 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0924 13:51:57.128788 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.149.252:37231: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260924 13:51:57.128904 3899 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.149.252:37231: Network error: Client connection negotiation failed: client connection to 127.13.149.252:37231: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.149.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.252.keytab.
Entry for principal kudu/127.13.149.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.252.keytab.
I20260924 13:51:57.136214 13911 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJN9WRF/build/release/bin/kudu
/tmp/dist-test-taskJN9WRF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790257629041645-13911-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790257629041645-13911-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790257629041645-13911-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790257629041645-13911-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.149.252:37231
--webserver_interface=127.13.149.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.252.keytab
--principal=kudu/127.13.149.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.149.212:36885
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.149.254:40929,127.13.149.253:36629,127.13.149.252:37231
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790257629041645-13911-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790257629041645-13911-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790257629041645-13911-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260924 13:51:57.175786 3824 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.149.253" port: 36629 } attempt: 3
I20260924 13:51:57.178223 3899 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.149.252" port: 37231 } attempt: 1
I20260924 13:51:57.178701 3901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:57.178527 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.252:37231 (local address 127.0.0.1:50900)
0924 13:51:57.178605 (+ 78us) negotiation.cc:107] Waiting for socket to connect
0924 13:51:57.178659 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.149.252:37231: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260924 13:51:57.179261 3907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:57.178953 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.252:37231 (local address 127.0.0.1:50914)
0924 13:51:57.179137 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0924 13:51:57.179174 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.149.252:37231: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":114,"threads_started":1}
W20260924 13:51:57.179606 3899 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.149.252:37231: Network error: Client connection negotiation failed: client connection to 127.13.149.252:37231: connect: Connection refused (error 111)
Sep 24 13:51:57 dist-test-slave-sjl2 krb5kdc[3743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257916, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.149.254@KRBTEST.COM for kudu/127.13.149.253@KRBTEST.COM
I20260924 13:51:57.183125 3903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:57.176346 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44728 (local address 127.13.149.253:36629)
0924 13:51:57.176419 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0924 13:51:57.176422 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 13:51:57.176435 (+ 13us) server_negotiation.cc:407] Connection header received
0924 13:51:57.176469 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:51:57.176470 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:51:57.176484 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:51:57.176547 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:51:57.176870 (+ 323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:51:57.177772 (+ 902us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:51:57.178675 (+ 903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:51:57.178960 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:51:57.179019 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:51:57.180874 (+ 1855us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:51:57.180875 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:51:57.180876 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 13:51:57.180883 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:51:57.182149 (+ 1266us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:51:57.182444 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:51:57.182445 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:51:57.182445 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:51:57.182498 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:51:57.182655 (+ 157us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:51:57.182655 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:51:57.182656 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:51:57.182764 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:51:57.182820 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0924 13:51:57.182961 (+ 141us) server_negotiation.cc:299] Negotiation successful
0924 13:51:57.183033 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260924 13:51:57.183128 3826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:57.176255 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.253:36629 (local address 127.0.0.1:44728)
0924 13:51:57.176345 (+ 90us) negotiation.cc:107] Waiting for socket to connect
0924 13:51:57.176358 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0924 13:51:57.176420 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:51:57.176563 (+ 143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:51:57.176564 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:51:57.176729 (+ 165us) client_negotiation.cc:189] Negotiated authn=SASL
0924 13:51:57.176798 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:51:57.176800 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:51:57.177894 (+ 1094us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:51:57.177895 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:51:57.178593 (+ 698us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:51:57.178597 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:51:57.179028 (+ 431us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:51:57.179030 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:51:57.179102 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:51:57.179569 (+ 467us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 13:51:57.179574 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 13:51:57.180780 (+ 1206us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 13:51:57.182226 (+ 1446us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 13:51:57.182227 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 13:51:57.182228 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 13:51:57.182385 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 13:51:57.182551 (+ 166us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 13:51:57.182551 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 13:51:57.182552 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 13:51:57.182606 (+ 54us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 13:51:57.182817 (+ 211us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 13:51:57.182818 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 13:51:57.182875 (+ 57us) client_negotiation.cc:769] Sending connection context
0924 13:51:57.182934 (+ 59us) client_negotiation.cc:240] Negotiation successful
0924 13:51:57.183033 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":60}
I20260924 13:51:57.183522 3824 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.149.252" port: 37231 } has no permanent_uuid. Determining permanent_uuid...
I20260924 13:51:57.183818 3826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:57.183744 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.252:37231 (local address 127.0.0.1:50928)
0924 13:51:57.183776 (+ 32us) negotiation.cc:107] Waiting for socket to connect
0924 13:51:57.183802 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.149.252:37231: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":19}
I20260924 13:51:57.184094 3826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:57.184017 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.252:37231 (local address 127.0.0.1:50938)
0924 13:51:57.184052 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0924 13:51:57.184074 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.149.252:37231: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
W20260924 13:51:57.184165 3824 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.149.252:37231: Network error: Client connection negotiation failed: client connection to 127.13.149.252:37231: connect: Connection refused (error 111)
I20260924 13:51:57.215332 3824 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.149.252" port: 37231 } attempt: 1
I20260924 13:51:57.215858 3826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:57.215712 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.252:37231 (local address 127.0.0.1:50952)
0924 13:51:57.215777 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0924 13:51:57.215817 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.149.252:37231: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260924 13:51:57.216156 3826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:57.216077 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.252:37231 (local address 127.0.0.1:50966)
0924 13:51:57.216115 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0924 13:51:57.216139 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.149.252:37231: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
W20260924 13:51:57.216233 3824 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.149.252:37231: Network error: Client connection negotiation failed: client connection to 127.13.149.252:37231: connect: Connection refused (error 111)
W20260924 13:51:57.221143 3906 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260924 13:51:57.221393 3906 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 13:51:57.221419 3906 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 13:51:57.221455 3906 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 13:51:57.222716 3906 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 13:51:57.222769 3906 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 13:51:57.222787 3906 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 13:51:57.222803 3906 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 13:51:57.222819 3906 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 13:51:57.224399 3906 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.149.212:36885
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790257629041645-13911-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790257629041645-13911-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.13.149.254:40929,127.13.149.253:36629,127.13.149.252:37231
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.252.keytab
--principal=kudu/127.13.149.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.149.252:37231
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790257629041645-13911-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.13.149.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3906
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790257629041645-13911-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 12:29:37 UTC on ccycloud.quasar-iqzbxn.root.comops.site
build id 4670106
I20260924 13:51:57.224651 3906 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 13:51:57.224880 3906 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 13:51:57.229671 3915 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 13:51:57.229664 3913 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 13:51:57.229691 3912 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 13:51:57.229897 3906 server_base.cc:1034] running on GCE node
I20260924 13:51:57.230118 3906 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 13:51:57.230383 3906 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 13:51:57.231528 3906 hybrid_clock.cc:648] HybridClock initialized: now 1790257917231509 us; error 30 us; skew 500 ppm
Sep 24 13:51:57 dist-test-slave-sjl2 krb5kdc[3743](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257917, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.149.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 13:51:57.234127 3906 init.cc:377] Logged in from keytab as kudu/127.13.149.252@KRBTEST.COM (short username kudu)
I20260924 13:51:57.234591 3906 webserver.cc:460] Webserver started at http://127.13.149.252:43197/ using document root <none> and password file <none>
I20260924 13:51:57.234802 3906 fs_manager.cc:359] Metadata directory not provided
I20260924 13:51:57.234849 3906 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 13:51:57.234977 3906 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 13:51:57.236048 3906 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790257629041645-13911-0/minicluster-data/master-2/data/instance:
uuid: "9b978cdeb87447bd8b32ac957d2a1a91"
format_stamp: "Formatted at 2026-09-24 13:51:57 on dist-test-slave-sjl2"
server_key: "d7ee92aa3590e86666fb138e22109b66"
server_key_iv: "a8545b9848181dade48cef613f2d9db4"
server_key_version: "encryptionkey@0"
I20260924 13:51:57.236388 3906 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790257629041645-13911-0/minicluster-data/master-2/wal/instance:
uuid: "9b978cdeb87447bd8b32ac957d2a1a91"
format_stamp: "Formatted at 2026-09-24 13:51:57 on dist-test-slave-sjl2"
server_key: "d7ee92aa3590e86666fb138e22109b66"
server_key_iv: "a8545b9848181dade48cef613f2d9db4"
server_key_version: "encryptionkey@0"
I20260924 13:51:57.237725 3906 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260924 13:51:57.238394 3922 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 13:51:57.238538 3906 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260924 13:51:57.238620 3906 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790257629041645-13911-0/minicluster-data/master-2/data,/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790257629041645-13911-0/minicluster-data/master-2/wal
uuid: "9b978cdeb87447bd8b32ac957d2a1a91"
format_stamp: "Formatted at 2026-09-24 13:51:57 on dist-test-slave-sjl2"
server_key: "d7ee92aa3590e86666fb138e22109b66"
server_key_iv: "a8545b9848181dade48cef613f2d9db4"
server_key_version: "encryptionkey@0"
I20260924 13:51:57.238703 3906 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790257629041645-13911-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790257629041645-13911-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790257629041645-13911-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 13:51:57.247784 3899 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.149.252" port: 37231 } attempt: 2
I20260924 13:51:57.248317 3907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:57.248177 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.252:37231 (local address 127.0.0.1:50980)
0924 13:51:57.248243 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0924 13:51:57.248284 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.149.252:37231: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260924 13:51:57.248665 3907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:57.248566 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.252:37231 (local address 127.0.0.1:50996)
0924 13:51:57.248609 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0924 13:51:57.248636 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.149.252:37231: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
W20260924 13:51:57.248754 3899 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.149.252:37231: Network error: Client connection negotiation failed: client connection to 127.13.149.252:37231: connect: Connection refused (error 111)
I20260924 13:51:57.268718 3906 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 13:51:57.271121 3906 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 13:51:57.271281 3906 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 13:51:57.275441 3906 rpc_server.cc:307] RPC server started. Bound to: 127.13.149.252:37231
I20260924 13:51:57.275473 3974 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.149.252:37231 every 8 connection(s)
I20260924 13:51:57.275841 3906 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790257629041645-13911-0/minicluster-data/master-2/data/info.pb
I20260924 13:51:57.276757 3975 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 13:51:57.278350 3975 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.149.254" port: 40929 } has no permanent_uuid. Determining permanent_uuid...
I20260924 13:51:57.281344 13911 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJN9WRF/build/release/bin/kudu as pid 3906
I20260924 13:51:57.281428 13911 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790257629041645-13911-0/minicluster-data/master-2/wal/instance
I20260924 13:51:57.281596 13911 external_mini_cluster.cc:1428] Setting key fdc4b8801fbac24c4cd139a4083ab14c
Sep 24 13:51:57 dist-test-slave-sjl2 krb5kdc[3743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257917, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.149.252@KRBTEST.COM for kudu/127.13.149.254@KRBTEST.COM
Sep 24 13:51:57 dist-test-slave-sjl2 krb5kdc[3743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257916, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.149.252@KRBTEST.COM
I20260924 13:51:57.288275 3827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:57.279095 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45784 (local address 127.13.149.254:40929)
0924 13:51:57.279275 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0924 13:51:57.279277 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:51:57.279389 (+ 112us) server_negotiation.cc:407] Connection header received
0924 13:51:57.279912 (+ 523us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:51:57.279913 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:51:57.279926 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:51:57.279974 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:51:57.280585 (+ 611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:51:57.281118 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:51:57.282477 (+ 1359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:51:57.282769 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:51:57.282859 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:51:57.285337 (+ 2478us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:51:57.285339 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:51:57.285339 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 13:51:57.285346 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:51:57.286803 (+ 1457us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:51:57.287243 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:51:57.287244 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:51:57.287245 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:51:57.287309 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:51:57.287583 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:51:57.287584 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:51:57.287585 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:51:57.287713 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:51:57.287764 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0924 13:51:57.288098 (+ 334us) server_negotiation.cc:299] Negotiation successful
0924 13:51:57.288183 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":105}
I20260924 13:51:57.288478 3977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:57.279045 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:40929 (local address 127.0.0.1:45784)
0924 13:51:57.279243 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0924 13:51:57.279293 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0924 13:51:57.279813 (+ 520us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:51:57.280047 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:51:57.280051 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:51:57.280303 (+ 252us) client_negotiation.cc:189] Negotiated authn=SASL
0924 13:51:57.280498 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:51:57.280501 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:51:57.281207 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:51:57.281208 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:51:57.282402 (+ 1194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:51:57.282404 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:51:57.282868 (+ 464us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:51:57.282869 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:51:57.282981 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:51:57.283484 (+ 503us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 13:51:57.283494 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0924 13:51:57.285230 (+ 1736us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 13:51:57.286918 (+ 1688us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 13:51:57.286927 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 13:51:57.286929 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 13:51:57.287168 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 13:51:57.287382 (+ 214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 13:51:57.287383 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 13:51:57.287383 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 13:51:57.287489 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 13:51:57.287773 (+ 284us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 13:51:57.287781 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 13:51:57.287954 (+ 173us) client_negotiation.cc:769] Sending connection context
0924 13:51:57.288059 (+ 105us) client_negotiation.cc:240] Negotiation successful
0924 13:51:57.288208 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":92,"threads_started":1}
I20260924 13:51:57.289077 3975 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.149.253" port: 36629 } has no permanent_uuid. Determining permanent_uuid...
I20260924 13:51:57.291076 3979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:57.282034 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51000 (local address 127.13.149.252:37231)
0924 13:51:57.282240 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0924 13:51:57.282245 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 13:51:57.282264 (+ 19us) server_negotiation.cc:407] Connection header received
0924 13:51:57.282286 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:51:57.282288 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:51:57.282307 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:51:57.282398 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:51:57.283128 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:51:57.284076 (+ 948us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:51:57.285070 (+ 994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:51:57.285448 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:51:57.285498 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:51:57.287990 (+ 2492us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:51:57.287997 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:51:57.287997 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 13:51:57.288004 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:51:57.289823 (+ 1819us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:51:57.290206 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:51:57.290208 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:51:57.290209 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:51:57.290270 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:51:57.290459 (+ 189us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:51:57.290460 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:51:57.290460 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:51:57.290650 (+ 190us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:51:57.290835 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0924 13:51:57.290879 (+ 44us) server_negotiation.cc:299] Negotiation successful
0924 13:51:57.290960 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":172,"thread_start_us":99,"threads_started":1}
I20260924 13:51:57.291379 3824 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.149.252" port: 37231 } attempt: 2
Sep 24 13:51:57 dist-test-slave-sjl2 krb5kdc[3743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257917, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.149.252@KRBTEST.COM for kudu/127.13.149.253@KRBTEST.COM
Sep 24 13:51:57 dist-test-slave-sjl2 krb5kdc[3743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257916, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.149.254@KRBTEST.COM for kudu/127.13.149.252@KRBTEST.COM
Sep 24 13:51:57 dist-test-slave-sjl2 krb5kdc[3743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257916, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.149.254@KRBTEST.COM
I20260924 13:51:57.297675 3977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:57.289343 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.253:36629 (local address 127.0.0.1:44738)
0924 13:51:57.289914 (+ 571us) negotiation.cc:107] Waiting for socket to connect
0924 13:51:57.289925 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0924 13:51:57.289967 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:51:57.290084 (+ 117us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:51:57.290084 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:51:57.290214 (+ 130us) client_negotiation.cc:189] Negotiated authn=SASL
0924 13:51:57.290258 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:51:57.290260 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:51:57.291076 (+ 816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:51:57.291077 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:51:57.292087 (+ 1010us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:51:57.292090 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:51:57.292418 (+ 328us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:51:57.292419 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:51:57.292474 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:51:57.293411 (+ 937us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 13:51:57.293417 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0924 13:51:57.294823 (+ 1406us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 13:51:57.296484 (+ 1661us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 13:51:57.296486 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 13:51:57.296486 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 13:51:57.296726 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 13:51:57.296977 (+ 251us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 13:51:57.296978 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 13:51:57.296978 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 13:51:57.297043 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 13:51:57.297348 (+ 305us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 13:51:57.297349 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 13:51:57.297395 (+ 46us) client_negotiation.cc:769] Sending connection context
0924 13:51:57.297462 (+ 67us) client_negotiation.cc:240] Negotiation successful
0924 13:51:57.297558 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":554}
I20260924 13:51:57.297701 3903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:57.289397 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44738 (local address 127.13.149.253:36629)
0924 13:51:57.289449 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0924 13:51:57.289452 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 13:51:57.289984 (+ 532us) server_negotiation.cc:407] Connection header received
0924 13:51:57.290015 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:51:57.290016 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:51:57.290027 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:51:57.290081 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:51:57.290334 (+ 253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:51:57.290996 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:51:57.292145 (+ 1149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:51:57.292362 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:51:57.292415 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:51:57.294894 (+ 2479us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:51:57.294895 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:51:57.294895 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 13:51:57.294904 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:51:57.296402 (+ 1498us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:51:57.296806 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:51:57.296807 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:51:57.296807 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:51:57.296870 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:51:57.297105 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:51:57.297106 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:51:57.297106 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:51:57.297255 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:51:57.297323 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0924 13:51:57.297514 (+ 191us) server_negotiation.cc:299] Negotiation successful
0924 13:51:57.297598 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260924 13:51:57.298307 3975 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.149.252" port: 37231 } has no permanent_uuid. Determining permanent_uuid...
I20260924 13:51:57.298420 3979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:57.291732 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51012 (local address 127.13.149.252:37231)
0924 13:51:57.291766 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0924 13:51:57.291768 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:51:57.291775 (+ 7us) server_negotiation.cc:407] Connection header received
0924 13:51:57.291819 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:51:57.291820 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:51:57.291829 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:51:57.291868 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:51:57.292154 (+ 286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:51:57.292849 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:51:57.293507 (+ 658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:51:57.293728 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:51:57.293779 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:51:57.295834 (+ 2055us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:51:57.295835 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:51:57.295835 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 13:51:57.295842 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:51:57.297248 (+ 1406us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:51:57.297643 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:51:57.297643 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:51:57.297644 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:51:57.297692 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:51:57.297908 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:51:57.297908 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:51:57.297908 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:51:57.298032 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:51:57.298090 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0924 13:51:57.298249 (+ 159us) server_negotiation.cc:299] Negotiation successful
0924 13:51:57.298329 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260924 13:51:57.298424 3826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:57.291673 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.252:37231 (local address 127.0.0.1:51012)
0924 13:51:57.291722 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0924 13:51:57.291733 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0924 13:51:57.291775 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:51:57.291881 (+ 106us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:51:57.291882 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:51:57.292032 (+ 150us) client_negotiation.cc:189] Negotiated authn=SASL
0924 13:51:57.292078 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:51:57.292080 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:51:57.292938 (+ 858us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:51:57.292940 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:51:57.293438 (+ 498us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:51:57.293439 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:51:57.293786 (+ 347us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:51:57.293787 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:51:57.293844 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:51:57.294203 (+ 359us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 13:51:57.294209 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0924 13:51:57.295574 (+ 1365us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 13:51:57.297349 (+ 1775us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 13:51:57.297351 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 13:51:57.297351 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 13:51:57.297565 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 13:51:57.297764 (+ 199us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 13:51:57.297765 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 13:51:57.297765 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 13:51:57.297842 (+ 77us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 13:51:57.298099 (+ 257us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 13:51:57.298100 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 13:51:57.298146 (+ 46us) client_negotiation.cc:769] Sending connection context
0924 13:51:57.298212 (+ 66us) client_negotiation.cc:240] Negotiation successful
0924 13:51:57.298342 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30}
I20260924 13:51:57.300390 3827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:57.292022 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45786 (local address 127.13.149.254:40929)
0924 13:51:57.292060 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0924 13:51:57.292062 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:51:57.292084 (+ 22us) server_negotiation.cc:407] Connection header received
0924 13:51:57.292099 (+ 15us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:51:57.292100 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:51:57.292108 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:51:57.292149 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:51:57.292764 (+ 615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:51:57.293292 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:51:57.294217 (+ 925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:51:57.294420 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:51:57.294475 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:51:57.297714 (+ 3239us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:51:57.297715 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:51:57.297716 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 13:51:57.297724 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:51:57.299291 (+ 1567us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:51:57.299679 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:51:57.299680 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:51:57.299680 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:51:57.299732 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:51:57.299902 (+ 170us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:51:57.299903 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:51:57.299903 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:51:57.300007 (+ 104us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:51:57.300177 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0924 13:51:57.300225 (+ 48us) server_negotiation.cc:299] Negotiation successful
0924 13:51:57.300302 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260924 13:51:57.300393 3824 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 252a98bd316f447c8b2f98ea1802a935: Bootstrap starting.
I20260924 13:51:57.301359 3824 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 252a98bd316f447c8b2f98ea1802a935: Neither blocks nor log segments found. Creating new log.
I20260924 13:51:57.301672 3824 log.cc:824] T 00000000000000000000000000000000 P 252a98bd316f447c8b2f98ea1802a935: Log is configured to *not* fsync() on all Append() calls
Sep 24 13:51:57 dist-test-slave-sjl2 krb5kdc[3743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257917, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.149.252@KRBTEST.COM for kudu/127.13.149.252@KRBTEST.COM
I20260924 13:51:57.302583 3824 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 252a98bd316f447c8b2f98ea1802a935: No bootstrap required, opened a new log
I20260924 13:51:57.304623 3824 raft_consensus.cc:348] T 00000000000000000000000000000000 P 252a98bd316f447c8b2f98ea1802a935 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "252a98bd316f447c8b2f98ea1802a935" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 40929 } } peers { permanent_uuid: "9c78d1aa2ebc4ea7899bcc2a1a0ef022" member_type: VOTER last_known_addr { host: "127.13.149.253" port: 36629 } } peers { permanent_uuid: "9b978cdeb87447bd8b32ac957d2a1a91" member_type: VOTER last_known_addr { host: "127.13.149.252" port: 37231 } }
I20260924 13:51:57.304801 3824 raft_consensus.cc:729] T 00000000000000000000000000000000 P 252a98bd316f447c8b2f98ea1802a935 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 252a98bd316f447c8b2f98ea1802a935, State: Initialized, Role: FOLLOWER
I20260924 13:51:57.304960 3824 consensus_queue.cc:260] T 00000000000000000000000000000000 P 252a98bd316f447c8b2f98ea1802a935 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "252a98bd316f447c8b2f98ea1802a935" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 40929 } } peers { permanent_uuid: "9c78d1aa2ebc4ea7899bcc2a1a0ef022" member_type: VOTER last_known_addr { host: "127.13.149.253" port: 36629 } } peers { permanent_uuid: "9b978cdeb87447bd8b32ac957d2a1a91" member_type: VOTER last_known_addr { host: "127.13.149.252" port: 37231 } }
I20260924 13:51:57.305298 3982 sys_catalog.cc:455] T 00000000000000000000000000000000 P 252a98bd316f447c8b2f98ea1802a935 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "252a98bd316f447c8b2f98ea1802a935" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 40929 } } peers { permanent_uuid: "9c78d1aa2ebc4ea7899bcc2a1a0ef022" member_type: VOTER last_known_addr { host: "127.13.149.253" port: 36629 } } peers { permanent_uuid: "9b978cdeb87447bd8b32ac957d2a1a91" member_type: VOTER last_known_addr { host: "127.13.149.252" port: 37231 } } }
I20260924 13:51:57.305383 3824 sys_catalog.cc:564] T 00000000000000000000000000000000 P 252a98bd316f447c8b2f98ea1802a935 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 13:51:57.305496 3982 sys_catalog.cc:458] T 00000000000000000000000000000000 P 252a98bd316f447c8b2f98ea1802a935 [sys.catalog]: This master's current role is: FOLLOWER
I20260924 13:51:57.305631 3977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:57.298562 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.252:37231 (local address 127.0.0.1:51024)
0924 13:51:57.298592 (+ 30us) negotiation.cc:107] Waiting for socket to connect
0924 13:51:57.298602 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 13:51:57.298635 (+ 33us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:51:57.298862 (+ 227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:51:57.298863 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:51:57.298995 (+ 132us) client_negotiation.cc:189] Negotiated authn=SASL
0924 13:51:57.299036 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:51:57.299037 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:51:57.299875 (+ 838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:51:57.299876 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:51:57.300565 (+ 689us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:51:57.300569 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:51:57.301047 (+ 478us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:51:57.301048 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:51:57.301086 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:51:57.301533 (+ 447us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 13:51:57.301538 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 13:51:57.302906 (+ 1368us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 13:51:57.304486 (+ 1580us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 13:51:57.304487 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 13:51:57.304487 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 13:51:57.304638 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 13:51:57.304961 (+ 323us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 13:51:57.304961 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 13:51:57.304962 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 13:51:57.305020 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 13:51:57.305345 (+ 325us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 13:51:57.305346 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 13:51:57.305386 (+ 40us) client_negotiation.cc:769] Sending connection context
0924 13:51:57.305429 (+ 43us) client_negotiation.cc:240] Negotiation successful
0924 13:51:57.305537 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13}
Sep 24 13:51:57 dist-test-slave-sjl2 krb5kdc[3743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257916, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.149.253@KRBTEST.COM
I20260924 13:51:57.305976 3979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:57.298603 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51024 (local address 127.13.149.252:37231)
0924 13:51:57.298739 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0924 13:51:57.298741 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:51:57.298748 (+ 7us) server_negotiation.cc:407] Connection header received
0924 13:51:57.298765 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:51:57.298766 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:51:57.298776 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:51:57.298827 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:51:57.299103 (+ 276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:51:57.299785 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:51:57.300640 (+ 855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:51:57.300901 (+ 261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:51:57.300990 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:51:57.302991 (+ 2001us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:51:57.302992 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:51:57.302992 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 13:51:57.302999 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:51:57.304419 (+ 1420us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:51:57.304702 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:51:57.304703 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:51:57.304703 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:51:57.304753 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:51:57.305073 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:51:57.305074 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:51:57.305075 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:51:57.305273 (+ 198us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:51:57.305801 (+ 528us) server_negotiation.cc:1035] Waiting for connection context
0924 13:51:57.305837 (+ 36us) server_negotiation.cc:299] Negotiation successful
0924 13:51:57.305909 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":121,"thread_start_us":109,"threads_started":1}
I20260924 13:51:57.307422 3975 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9b978cdeb87447bd8b32ac957d2a1a91: Bootstrap starting.
W20260924 13:51:57.307823 3993 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 252a98bd316f447c8b2f98ea1802a935: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260924 13:51:57.307869 3993 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260924 13:51:57.308423 3975 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9b978cdeb87447bd8b32ac957d2a1a91: Neither blocks nor log segments found. Creating new log.
I20260924 13:51:57.308737 3975 log.cc:824] T 00000000000000000000000000000000 P 9b978cdeb87447bd8b32ac957d2a1a91: Log is configured to *not* fsync() on all Append() calls
I20260924 13:51:57.309337 3903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:57.301179 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44748 (local address 127.13.149.253:36629)
0924 13:51:57.301453 (+ 274us) server_negotiation.cc:206] Beginning negotiation
0924 13:51:57.301454 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0924 13:51:57.301542 (+ 88us) server_negotiation.cc:407] Connection header received
0924 13:51:57.301598 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:51:57.301599 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:51:57.301609 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:51:57.301655 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:51:57.302217 (+ 562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:51:57.302720 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:51:57.303525 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:51:57.303806 (+ 281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:51:57.303874 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:51:57.306411 (+ 2537us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:51:57.306412 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:51:57.306413 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 13:51:57.306420 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:51:57.308010 (+ 1590us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:51:57.308406 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:51:57.308407 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:51:57.308408 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:51:57.308486 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:51:57.308687 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:51:57.308688 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:51:57.308688 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:51:57.308818 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:51:57.308993 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0924 13:51:57.309173 (+ 180us) server_negotiation.cc:299] Negotiation successful
0924 13:51:57.309252 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260924 13:51:57.310487 13911 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260924 13:51:57.308346 3756 master_runner.cc:186] Error getting master registration for 127.13.149.253:36629: OK, instance_id { permanent_uuid: "9c78d1aa2ebc4ea7899bcc2a1a0ef022" instance_seqno: 1790257917105692 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260924 13:51:57.311219 3756 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260924 13:51:57.311422 3798 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "252a98bd316f447c8b2f98ea1802a935"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45786
I20260924 13:51:57.311539 3798 raft_consensus.cc:482] T 00000000000000000000000000000000 P 252a98bd316f447c8b2f98ea1802a935 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260924 13:51:57.311590 3798 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 252a98bd316f447c8b2f98ea1802a935 [term 0 FOLLOWER]: Advancing to term 1
I20260924 13:51:57.311944 3975 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9b978cdeb87447bd8b32ac957d2a1a91: No bootstrap required, opened a new log
I20260924 13:51:57.312249 3798 raft_consensus.cc:504] T 00000000000000000000000000000000 P 252a98bd316f447c8b2f98ea1802a935 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "252a98bd316f447c8b2f98ea1802a935" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 40929 } } peers { permanent_uuid: "9c78d1aa2ebc4ea7899bcc2a1a0ef022" member_type: VOTER last_known_addr { host: "127.13.149.253" port: 36629 } } peers { permanent_uuid: "9b978cdeb87447bd8b32ac957d2a1a91" member_type: VOTER last_known_addr { host: "127.13.149.252" port: 37231 } }
I20260924 13:51:57.312430 3798 leader_election.cc:288] T 00000000000000000000000000000000 P 252a98bd316f447c8b2f98ea1802a935 [CANDIDATE]: Term 1 election: Requested vote from peers 9c78d1aa2ebc4ea7899bcc2a1a0ef022 (127.13.149.253:36629), 9b978cdeb87447bd8b32ac957d2a1a91 (127.13.149.252:37231)
I20260924 13:51:57.314463 3975 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9b978cdeb87447bd8b32ac957d2a1a91 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "252a98bd316f447c8b2f98ea1802a935" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 40929 } } peers { permanent_uuid: "9c78d1aa2ebc4ea7899bcc2a1a0ef022" member_type: VOTER last_known_addr { host: "127.13.149.253" port: 36629 } } peers { permanent_uuid: "9b978cdeb87447bd8b32ac957d2a1a91" member_type: VOTER last_known_addr { host: "127.13.149.252" port: 37231 } }
I20260924 13:51:57.314637 3975 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9b978cdeb87447bd8b32ac957d2a1a91 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9b978cdeb87447bd8b32ac957d2a1a91, State: Initialized, Role: FOLLOWER
I20260924 13:51:57.314733 3975 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9b978cdeb87447bd8b32ac957d2a1a91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "252a98bd316f447c8b2f98ea1802a935" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 40929 } } peers { permanent_uuid: "9c78d1aa2ebc4ea7899bcc2a1a0ef022" member_type: VOTER last_known_addr { host: "127.13.149.253" port: 36629 } } peers { permanent_uuid: "9b978cdeb87447bd8b32ac957d2a1a91" member_type: VOTER last_known_addr { host: "127.13.149.252" port: 37231 } }
I20260924 13:51:57.315073 3975 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9b978cdeb87447bd8b32ac957d2a1a91 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 13:51:57.315867 3949 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "252a98bd316f447c8b2f98ea1802a935" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "9b978cdeb87447bd8b32ac957d2a1a91"
I20260924 13:51:57.316020 3949 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9b978cdeb87447bd8b32ac957d2a1a91 [term 0 FOLLOWER]: Advancing to term 1
I20260924 13:51:57.316990 3949 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 9b978cdeb87447bd8b32ac957d2a1a91 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 252a98bd316f447c8b2f98ea1802a935 in term 1.
I20260924 13:51:57.317344 3773 leader_election.cc:302] T 00000000000000000000000000000000 P 252a98bd316f447c8b2f98ea1802a935 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 252a98bd316f447c8b2f98ea1802a935, 9b978cdeb87447bd8b32ac957d2a1a91; no voters:
I20260924 13:51:57.317565 3982 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 252a98bd316f447c8b2f98ea1802a935 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 13:51:57.317766 3982 raft_consensus.cc:686] T 00000000000000000000000000000000 P 252a98bd316f447c8b2f98ea1802a935 [term 1 LEADER]: Becoming Leader. State: Replica: 252a98bd316f447c8b2f98ea1802a935, State: Running, Role: LEADER
I20260924 13:51:57.317876 3982 consensus_queue.cc:237] T 00000000000000000000000000000000 P 252a98bd316f447c8b2f98ea1802a935 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "252a98bd316f447c8b2f98ea1802a935" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 40929 } } peers { permanent_uuid: "9c78d1aa2ebc4ea7899bcc2a1a0ef022" member_type: VOTER last_known_addr { host: "127.13.149.253" port: 36629 } } peers { permanent_uuid: "9b978cdeb87447bd8b32ac957d2a1a91" member_type: VOTER last_known_addr { host: "127.13.149.252" port: 37231 } }
I20260924 13:51:57.318832 3982 sys_catalog.cc:455] T 00000000000000000000000000000000 P 252a98bd316f447c8b2f98ea1802a935 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 252a98bd316f447c8b2f98ea1802a935. Latest consensus state: current_term: 1 leader_uuid: "252a98bd316f447c8b2f98ea1802a935" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "252a98bd316f447c8b2f98ea1802a935" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 40929 } } peers { permanent_uuid: "9c78d1aa2ebc4ea7899bcc2a1a0ef022" member_type: VOTER last_known_addr { host: "127.13.149.253" port: 36629 } } peers { permanent_uuid: "9b978cdeb87447bd8b32ac957d2a1a91" member_type: VOTER last_known_addr { host: "127.13.149.252" port: 37231 } } }
I20260924 13:51:57.318958 3982 sys_catalog.cc:458] T 00000000000000000000000000000000 P 252a98bd316f447c8b2f98ea1802a935 [sys.catalog]: This master's current role is: LEADER
I20260924 13:51:57.321026 4002 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9b978cdeb87447bd8b32ac957d2a1a91 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "252a98bd316f447c8b2f98ea1802a935" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 40929 } } peers { permanent_uuid: "9c78d1aa2ebc4ea7899bcc2a1a0ef022" member_type: VOTER last_known_addr { host: "127.13.149.253" port: 36629 } } peers { permanent_uuid: "9b978cdeb87447bd8b32ac957d2a1a91" member_type: VOTER last_known_addr { host: "127.13.149.252" port: 37231 } } }
I20260924 13:51:57.322168 3903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:57.313610 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44752 (local address 127.13.149.253:36629)
0924 13:51:57.313664 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0924 13:51:57.313667 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 13:51:57.313920 (+ 253us) server_negotiation.cc:407] Connection header received
0924 13:51:57.314025 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:51:57.314026 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:51:57.314037 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:51:57.314089 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:51:57.314930 (+ 841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:51:57.315691 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:51:57.317332 (+ 1641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:51:57.317620 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:51:57.317684 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:51:57.319361 (+ 1677us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:51:57.319362 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:51:57.319363 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 13:51:57.319371 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:51:57.320955 (+ 1584us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:51:57.321348 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:51:57.321349 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:51:57.321350 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:51:57.321426 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:51:57.321607 (+ 181us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:51:57.321608 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:51:57.321609 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:51:57.321755 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:51:57.321855 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0924 13:51:57.322003 (+ 148us) server_negotiation.cc:299] Negotiation successful
0924 13:51:57.322080 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260924 13:51:57.321455 3906 master_runner.cc:186] Error getting master registration for 127.13.149.253:36629: OK, instance_id { permanent_uuid: "9c78d1aa2ebc4ea7899bcc2a1a0ef022" instance_seqno: 1790257917105692 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260924 13:51:57.324507 3906 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260924 13:51:57.321139 4002 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9b978cdeb87447bd8b32ac957d2a1a91 [sys.catalog]: This master's current role is: FOLLOWER
I20260924 13:51:57.327348 3979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:57.313784 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51032 (local address 127.13.149.252:37231)
0924 13:51:57.313824 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0924 13:51:57.313826 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:51:57.316778 (+ 2952us) server_negotiation.cc:407] Connection header received
0924 13:51:57.316881 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:51:57.316882 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:51:57.316893 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:51:57.316940 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:51:57.321130 (+ 4190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:51:57.321837 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:51:57.322796 (+ 959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:51:57.323117 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:51:57.323188 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:51:57.324582 (+ 1394us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:51:57.324583 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:51:57.324583 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 13:51:57.324591 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:51:57.326102 (+ 1511us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:51:57.326498 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:51:57.326499 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:51:57.326500 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:51:57.326567 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:51:57.326792 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:51:57.326809 (+ 17us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:51:57.326809 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:51:57.326944 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:51:57.327045 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0924 13:51:57.327179 (+ 134us) server_negotiation.cc:299] Negotiation successful
0924 13:51:57.327259 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260924 13:51:57.329142 4016 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 13:51:57.329402 4016 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 13:51:57.330219 3827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:57.323055 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45800 (local address 127.13.149.254:40929)
0924 13:51:57.323097 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0924 13:51:57.323099 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:51:57.323173 (+ 74us) server_negotiation.cc:407] Connection header received
0924 13:51:57.323264 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:51:57.323265 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:51:57.323276 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:51:57.323304 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:51:57.323943 (+ 639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:51:57.324603 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:51:57.325615 (+ 1012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:51:57.325916 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:51:57.325956 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:51:57.327455 (+ 1499us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:51:57.327457 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:51:57.327457 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 13:51:57.327465 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:51:57.329034 (+ 1569us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:51:57.329399 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:51:57.329401 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:51:57.329401 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:51:57.329473 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:51:57.329668 (+ 195us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:51:57.329669 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:51:57.329669 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:51:57.329815 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:51:57.329916 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0924 13:51:57.330051 (+ 135us) server_negotiation.cc:299] Negotiation successful
0924 13:51:57.330133 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260924 13:51:57.331305 3949 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 9b978cdeb87447bd8b32ac957d2a1a91 [term 1 FOLLOWER]: Refusing update from remote peer 252a98bd316f447c8b2f98ea1802a935: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 13:51:57.331705 4014 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 252a98bd316f447c8b2f98ea1802a935 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9b978cdeb87447bd8b32ac957d2a1a91" member_type: VOTER last_known_addr { host: "127.13.149.252" port: 37231 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260924 13:51:57.332160 4013 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 9b978cdeb87447bd8b32ac957d2a1a91: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260924 13:51:57.332212 4013 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260924 13:51:57.333329 4002 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9b978cdeb87447bd8b32ac957d2a1a91 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 252a98bd316f447c8b2f98ea1802a935. Latest consensus state: current_term: 1 leader_uuid: "252a98bd316f447c8b2f98ea1802a935" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "252a98bd316f447c8b2f98ea1802a935" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 40929 } } peers { permanent_uuid: "9c78d1aa2ebc4ea7899bcc2a1a0ef022" member_type: VOTER last_known_addr { host: "127.13.149.253" port: 36629 } } peers { permanent_uuid: "9b978cdeb87447bd8b32ac957d2a1a91" member_type: VOTER last_known_addr { host: "127.13.149.252" port: 37231 } } }
I20260924 13:51:57.333446 4002 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9b978cdeb87447bd8b32ac957d2a1a91 [sys.catalog]: This master's current role is: FOLLOWER
I20260924 13:51:57.334599 4014 sys_catalog.cc:455] T 00000000000000000000000000000000 P 252a98bd316f447c8b2f98ea1802a935 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "252a98bd316f447c8b2f98ea1802a935" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "252a98bd316f447c8b2f98ea1802a935" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 40929 } } peers { permanent_uuid: "9c78d1aa2ebc4ea7899bcc2a1a0ef022" member_type: VOTER last_known_addr { host: "127.13.149.253" port: 36629 } } peers { permanent_uuid: "9b978cdeb87447bd8b32ac957d2a1a91" member_type: VOTER last_known_addr { host: "127.13.149.252" port: 37231 } } }
I20260924 13:51:57.334713 4014 sys_catalog.cc:458] T 00000000000000000000000000000000 P 252a98bd316f447c8b2f98ea1802a935 [sys.catalog]: This master's current role is: LEADER
I20260924 13:51:57.335176 4002 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9b978cdeb87447bd8b32ac957d2a1a91 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "252a98bd316f447c8b2f98ea1802a935" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "252a98bd316f447c8b2f98ea1802a935" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 40929 } } peers { permanent_uuid: "9c78d1aa2ebc4ea7899bcc2a1a0ef022" member_type: VOTER last_known_addr { host: "127.13.149.253" port: 36629 } } peers { permanent_uuid: "9b978cdeb87447bd8b32ac957d2a1a91" member_type: VOTER last_known_addr { host: "127.13.149.252" port: 37231 } } }
I20260924 13:51:57.335261 4002 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9b978cdeb87447bd8b32ac957d2a1a91 [sys.catalog]: This master's current role is: FOLLOWER
I20260924 13:51:57.335507 4016 catalog_manager.cc:1347] Generated new cluster ID: 064fdbd66e4643518d36e55dbdac7145
I20260924 13:51:57.335549 4016 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 13:51:57.337052 4019 mvcc.cc:205] Tried to move back new op lower bound from 7332896429386366976 to 7332896429336711168. Current Snapshot: MvccSnapshot[applied={T|T < 7332896429386366976}]
I20260924 13:51:57.339877 3899 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.149.252" port: 37231 } attempt: 3
Sep 24 13:51:57 dist-test-slave-sjl2 krb5kdc[3743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257917, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.149.253@KRBTEST.COM for kudu/127.13.149.252@KRBTEST.COM
I20260924 13:51:57.349233 3979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:57.340432 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51046 (local address 127.13.149.252:37231)
0924 13:51:57.340520 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0924 13:51:57.340522 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:51:57.340529 (+ 7us) server_negotiation.cc:407] Connection header received
0924 13:51:57.340547 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:51:57.340547 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:51:57.340556 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:51:57.340602 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:51:57.341160 (+ 558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:51:57.341900 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:51:57.343003 (+ 1103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:51:57.343307 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:51:57.343373 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:51:57.345827 (+ 2454us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:51:57.345829 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:51:57.345829 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 13:51:57.345838 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:51:57.347605 (+ 1767us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:51:57.348015 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:51:57.348016 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:51:57.348016 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:51:57.348077 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:51:57.348400 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:51:57.348401 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:51:57.348401 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:51:57.348579 (+ 178us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:51:57.348657 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0924 13:51:57.348990 (+ 333us) server_negotiation.cc:299] Negotiation successful
0924 13:51:57.349071 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260924 13:51:57.349459 3907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:57.340207 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.252:37231 (local address 127.0.0.1:51046)
0924 13:51:57.340249 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0924 13:51:57.340261 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0924 13:51:57.340310 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:51:57.340655 (+ 345us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:51:57.340656 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:51:57.340863 (+ 207us) client_negotiation.cc:189] Negotiated authn=SASL
0924 13:51:57.341072 (+ 209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:51:57.341075 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:51:57.341990 (+ 915us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:51:57.341991 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:51:57.342890 (+ 899us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:51:57.342893 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:51:57.343390 (+ 497us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:51:57.343391 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:51:57.343476 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:51:57.344042 (+ 566us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 13:51:57.344048 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0924 13:51:57.345719 (+ 1671us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 13:51:57.347694 (+ 1975us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 13:51:57.347696 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 13:51:57.347696 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 13:51:57.347938 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 13:51:57.348248 (+ 310us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 13:51:57.348249 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 13:51:57.348250 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 13:51:57.348323 (+ 73us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 13:51:57.348647 (+ 324us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 13:51:57.348648 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 13:51:57.348700 (+ 52us) client_negotiation.cc:769] Sending connection context
0924 13:51:57.348765 (+ 65us) client_negotiation.cc:240] Negotiation successful
0924 13:51:57.348889 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":20}
I20260924 13:51:57.350996 3899 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9c78d1aa2ebc4ea7899bcc2a1a0ef022: Bootstrap starting.
I20260924 13:51:57.351740 3899 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9c78d1aa2ebc4ea7899bcc2a1a0ef022: Neither blocks nor log segments found. Creating new log.
I20260924 13:51:57.352074 3899 log.cc:824] T 00000000000000000000000000000000 P 9c78d1aa2ebc4ea7899bcc2a1a0ef022: Log is configured to *not* fsync() on all Append() calls
I20260924 13:51:57.354017 4016 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 13:51:57.354770 4016 catalog_manager.cc:1504] Loading token signing keys...
I20260924 13:51:57.361665 3899 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9c78d1aa2ebc4ea7899bcc2a1a0ef022: No bootstrap required, opened a new log
I20260924 13:51:57.363767 3899 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9c78d1aa2ebc4ea7899bcc2a1a0ef022 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "252a98bd316f447c8b2f98ea1802a935" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 40929 } } peers { permanent_uuid: "9c78d1aa2ebc4ea7899bcc2a1a0ef022" member_type: VOTER last_known_addr { host: "127.13.149.253" port: 36629 } } peers { permanent_uuid: "9b978cdeb87447bd8b32ac957d2a1a91" member_type: VOTER last_known_addr { host: "127.13.149.252" port: 37231 } }
I20260924 13:51:57.363934 3899 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9c78d1aa2ebc4ea7899bcc2a1a0ef022 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c78d1aa2ebc4ea7899bcc2a1a0ef022, State: Initialized, Role: FOLLOWER
I20260924 13:51:57.364008 3899 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9c78d1aa2ebc4ea7899bcc2a1a0ef022 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "252a98bd316f447c8b2f98ea1802a935" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 40929 } } peers { permanent_uuid: "9c78d1aa2ebc4ea7899bcc2a1a0ef022" member_type: VOTER last_known_addr { host: "127.13.149.253" port: 36629 } } peers { permanent_uuid: "9b978cdeb87447bd8b32ac957d2a1a91" member_type: VOTER last_known_addr { host: "127.13.149.252" port: 37231 } }
I20260924 13:51:57.364348 3899 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9c78d1aa2ebc4ea7899bcc2a1a0ef022 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 13:51:57.365236 3872 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9c78d1aa2ebc4ea7899bcc2a1a0ef022 [term 0 FOLLOWER]: Advancing to term 1
I20260924 13:51:57.366539 3872 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 9c78d1aa2ebc4ea7899bcc2a1a0ef022 [term 1 FOLLOWER]: Refusing update from remote peer 252a98bd316f447c8b2f98ea1802a935: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 13:51:57.366891 4014 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 252a98bd316f447c8b2f98ea1802a935 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9c78d1aa2ebc4ea7899bcc2a1a0ef022" member_type: VOTER last_known_addr { host: "127.13.149.253" port: 36629 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 13:51:57.367324 3873 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "252a98bd316f447c8b2f98ea1802a935" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "9c78d1aa2ebc4ea7899bcc2a1a0ef022"
I20260924 13:51:57.368146 3873 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 9c78d1aa2ebc4ea7899bcc2a1a0ef022 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 252a98bd316f447c8b2f98ea1802a935 in term 1.
W20260924 13:51:57.371937 4039 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 9c78d1aa2ebc4ea7899bcc2a1a0ef022: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260924 13:51:57.372062 4039 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260924 13:51:57.372983 4031 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9c78d1aa2ebc4ea7899bcc2a1a0ef022 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "252a98bd316f447c8b2f98ea1802a935" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "252a98bd316f447c8b2f98ea1802a935" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 40929 } } peers { permanent_uuid: "9c78d1aa2ebc4ea7899bcc2a1a0ef022" member_type: VOTER last_known_addr { host: "127.13.149.253" port: 36629 } } peers { permanent_uuid: "9b978cdeb87447bd8b32ac957d2a1a91" member_type: VOTER last_known_addr { host: "127.13.149.252" port: 37231 } } }
I20260924 13:51:57.373142 4031 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9c78d1aa2ebc4ea7899bcc2a1a0ef022 [sys.catalog]: This master's current role is: FOLLOWER
I20260924 13:51:57.373315 4031 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9c78d1aa2ebc4ea7899bcc2a1a0ef022 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 252a98bd316f447c8b2f98ea1802a935. Latest consensus state: current_term: 1 leader_uuid: "252a98bd316f447c8b2f98ea1802a935" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "252a98bd316f447c8b2f98ea1802a935" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 40929 } } peers { permanent_uuid: "9c78d1aa2ebc4ea7899bcc2a1a0ef022" member_type: VOTER last_known_addr { host: "127.13.149.253" port: 36629 } } peers { permanent_uuid: "9b978cdeb87447bd8b32ac957d2a1a91" member_type: VOTER last_known_addr { host: "127.13.149.252" port: 37231 } } }
I20260924 13:51:57.373415 4031 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9c78d1aa2ebc4ea7899bcc2a1a0ef022 [sys.catalog]: This master's current role is: FOLLOWER
I20260924 13:51:57.373484 4031 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9c78d1aa2ebc4ea7899bcc2a1a0ef022 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "252a98bd316f447c8b2f98ea1802a935" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "252a98bd316f447c8b2f98ea1802a935" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 40929 } } peers { permanent_uuid: "9c78d1aa2ebc4ea7899bcc2a1a0ef022" member_type: VOTER last_known_addr { host: "127.13.149.253" port: 36629 } } peers { permanent_uuid: "9b978cdeb87447bd8b32ac957d2a1a91" member_type: VOTER last_known_addr { host: "127.13.149.252" port: 37231 } } }
I20260924 13:51:57.373574 4031 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9c78d1aa2ebc4ea7899bcc2a1a0ef022 [sys.catalog]: This master's current role is: FOLLOWER
I20260924 13:51:57.374328 4029 mvcc.cc:205] Tried to move back new op lower bound from 7332896429477658624 to 7332896429336711168. Current Snapshot: MvccSnapshot[applied={T|T < 7332896429386366976}]
I20260924 13:51:57.376118 4014 sys_catalog.cc:455] T 00000000000000000000000000000000 P 252a98bd316f447c8b2f98ea1802a935 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "252a98bd316f447c8b2f98ea1802a935" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "252a98bd316f447c8b2f98ea1802a935" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 40929 } } peers { permanent_uuid: "9c78d1aa2ebc4ea7899bcc2a1a0ef022" member_type: VOTER last_known_addr { host: "127.13.149.253" port: 36629 } } peers { permanent_uuid: "9b978cdeb87447bd8b32ac957d2a1a91" member_type: VOTER last_known_addr { host: "127.13.149.252" port: 37231 } } }
I20260924 13:51:57.376236 4014 sys_catalog.cc:458] T 00000000000000000000000000000000 P 252a98bd316f447c8b2f98ea1802a935 [sys.catalog]: This master's current role is: LEADER
I20260924 13:51:57.377434 4016 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 252a98bd316f447c8b2f98ea1802a935: Generated new TSK 0
I20260924 13:51:57.377631 4016 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260924 13:51:57.402088 3827 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 13:51:57.400212 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45806 (local address 127.13.149.254:40929)
0924 13:51:57.400267 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0924 13:51:57.400269 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:51:57.401294 (+ 1025us) server_negotiation.cc:407] Connection header received
0924 13:51:57.401417 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:51:57.401419 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:51:57.401432 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:51:57.401466 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:51:57.402004 (+ 538us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45806: BlockingRecv error: recv got EOF from 127.0.0.1:45806 (error 108)
Metrics: {"server-negotiator.queue_time_us":29}
W20260924 13:51:57.403154 3903 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 13:51:57.400716 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44756 (local address 127.13.149.253:36629)
0924 13:51:57.400759 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0924 13:51:57.400761 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:51:57.402539 (+ 1778us) server_negotiation.cc:407] Connection header received
0924 13:51:57.402664 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:51:57.402665 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:51:57.402677 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:51:57.402710 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:51:57.403083 (+ 373us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44756: BlockingRecv error: recv got EOF from 127.0.0.1:44756 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260924 13:51:57.404268 3979 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 13:51:57.401098 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51048 (local address 127.13.149.252:37231)
0924 13:51:57.401151 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0924 13:51:57.401153 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:51:57.403653 (+ 2500us) server_negotiation.cc:407] Connection header received
0924 13:51:57.403799 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:51:57.403800 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:51:57.403814 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:51:57.403847 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:51:57.404185 (+ 338us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51048: BlockingRecv error: recv got EOF from 127.0.0.1:51048 (error 108)
Metrics: {"server-negotiator.queue_time_us":29}
W20260924 13:51:57.407260 3903 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 13:51:57.406371 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44766 (local address 127.13.149.253:36629)
0924 13:51:57.406419 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0924 13:51:57.406421 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:51:57.406650 (+ 229us) server_negotiation.cc:407] Connection header received
0924 13:51:57.406788 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:51:57.406790 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:51:57.406811 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:51:57.406845 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:51:57.407204 (+ 359us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44766: BlockingRecv error: recv got EOF from 127.0.0.1:44766 (error 108)
Metrics: {"server-negotiator.queue_time_us":26}
W20260924 13:51:57.410729 3903 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 13:51:57.409869 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44778 (local address 127.13.149.253:36629)
0924 13:51:57.409909 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0924 13:51:57.409911 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:51:57.410162 (+ 251us) server_negotiation.cc:407] Connection header received
0924 13:51:57.410269 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:51:57.410270 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:51:57.410281 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:51:57.410314 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:51:57.410674 (+ 360us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44778: BlockingRecv error: recv got EOF from 127.0.0.1:44778 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260924 13:51:57.415252 3903 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 13:51:57.414307 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44786 (local address 127.13.149.253:36629)
0924 13:51:57.414353 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0924 13:51:57.414355 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:51:57.414640 (+ 285us) server_negotiation.cc:407] Connection header received
0924 13:51:57.414755 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:51:57.414756 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:51:57.414769 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:51:57.414802 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:51:57.415192 (+ 390us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44786: BlockingRecv error: recv got EOF from 127.0.0.1:44786 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260924 13:51:57.421797 3903 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 13:51:57.420801 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44788 (local address 127.13.149.253:36629)
0924 13:51:57.420850 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0924 13:51:57.420852 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:51:57.421142 (+ 290us) server_negotiation.cc:407] Connection header received
0924 13:51:57.421256 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:51:57.421257 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:51:57.421269 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:51:57.421303 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:51:57.421733 (+ 430us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44788: BlockingRecv error: recv got EOF from 127.0.0.1:44788 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
W20260924 13:51:57.432453 3903 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 13:51:57.431415 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44798 (local address 127.13.149.253:36629)
0924 13:51:57.431465 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0924 13:51:57.431467 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:51:57.431791 (+ 324us) server_negotiation.cc:407] Connection header received
0924 13:51:57.431908 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:51:57.431909 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:51:57.431923 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:51:57.431958 (+ 35us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:51:57.432373 (+ 415us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44798: BlockingRecv error: recv got EOF from 127.0.0.1:44798 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
W20260924 13:51:57.451071 3903 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 13:51:57.450091 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44810 (local address 127.13.149.253:36629)
0924 13:51:57.450136 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0924 13:51:57.450138 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:51:57.450430 (+ 292us) server_negotiation.cc:407] Connection header received
0924 13:51:57.450549 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:51:57.450550 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:51:57.450562 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:51:57.450591 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:51:57.451007 (+ 416us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44810: BlockingRecv error: recv got EOF from 127.0.0.1:44810 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260924 13:51:57.485769 3903 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 13:51:57.484685 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44818 (local address 127.13.149.253:36629)
0924 13:51:57.484734 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0924 13:51:57.484736 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:51:57.485102 (+ 366us) server_negotiation.cc:407] Connection header received
0924 13:51:57.485224 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:51:57.485226 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:51:57.485240 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:51:57.485271 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:51:57.485688 (+ 417us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44818: BlockingRecv error: recv got EOF from 127.0.0.1:44818 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
W20260924 13:51:57.551905 3903 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 13:51:57.551260 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44828 (local address 127.13.149.253:36629)
0924 13:51:57.551329 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0924 13:51:57.551331 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:51:57.551407 (+ 76us) server_negotiation.cc:407] Connection header received
0924 13:51:57.551437 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:51:57.551438 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:51:57.551449 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:51:57.551515 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:51:57.551839 (+ 324us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44828: BlockingRecv error: recv got EOF from 127.0.0.1:44828 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
W20260924 13:51:57.681808 3903 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 13:51:57.681065 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44832 (local address 127.13.149.253:36629)
0924 13:51:57.681138 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0924 13:51:57.681140 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:51:57.681199 (+ 59us) server_negotiation.cc:407] Connection header received
0924 13:51:57.681265 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:51:57.681266 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:51:57.681282 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:51:57.681350 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:51:57.681747 (+ 397us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44832: BlockingRecv error: recv got EOF from 127.0.0.1:44832 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
W20260924 13:51:57.939762 3903 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 13:51:57.939096 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44848 (local address 127.13.149.253:36629)
0924 13:51:57.939172 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0924 13:51:57.939174 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:51:57.939210 (+ 36us) server_negotiation.cc:407] Connection header received
0924 13:51:57.939287 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:51:57.939288 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:51:57.939299 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:51:57.939356 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:51:57.939691 (+ 335us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44848: BlockingRecv error: recv got EOF from 127.0.0.1:44848 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
I20260924 13:51:58.332670 4013 catalog_manager.cc:1259] Loaded cluster ID: 064fdbd66e4643518d36e55dbdac7145
I20260924 13:51:58.332752 4013 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 9b978cdeb87447bd8b32ac957d2a1a91: loading cluster ID for follower catalog manager: success
I20260924 13:51:58.333643 4013 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 9b978cdeb87447bd8b32ac957d2a1a91: acquiring CA information for follower catalog manager: success
I20260924 13:51:58.333985 4013 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 9b978cdeb87447bd8b32ac957d2a1a91: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260924 13:51:58.372556 4039 catalog_manager.cc:1259] Loaded cluster ID: 064fdbd66e4643518d36e55dbdac7145
I20260924 13:51:58.372638 4039 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 9c78d1aa2ebc4ea7899bcc2a1a0ef022: loading cluster ID for follower catalog manager: success
I20260924 13:51:58.373497 4039 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 9c78d1aa2ebc4ea7899bcc2a1a0ef022: acquiring CA information for follower catalog manager: success
I20260924 13:51:58.373817 4039 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 9c78d1aa2ebc4ea7899bcc2a1a0ef022: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260924 13:51:58.455850 4108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:58.452949 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44864 (local address 127.13.149.253:36629)
0924 13:51:58.453055 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0924 13:51:58.453056 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0924 13:51:58.453071 (+ 15us) server_negotiation.cc:407] Connection header received
0924 13:51:58.453185 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:51:58.453186 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:51:58.453286 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:51:58.453368 (+ 82us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 13:51:58.453556 (+ 188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:51:58.454071 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:51:58.454947 (+ 876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:51:58.455169 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:51:58.455275 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:51:58.455517 (+ 242us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 13:51:58.455644 (+ 127us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 13:51:58.455721 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0924 13:51:58.455734 (+ 13us) server_negotiation.cc:299] Negotiation successful
0924 13:51:58.455756 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88,"thread_start_us":57,"threads_started":1}
I20260924 13:51:58.460170 4114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:58.457919 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51064 (local address 127.13.149.252:37231)
0924 13:51:58.458018 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0924 13:51:58.458020 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:51:58.458029 (+ 9us) server_negotiation.cc:407] Connection header received
0924 13:51:58.458099 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:51:58.458100 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:51:58.458137 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:51:58.458204 (+ 67us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 13:51:58.458283 (+ 79us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:51:58.458767 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:51:58.459505 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:51:58.459696 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:51:58.459742 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:51:58.459841 (+ 99us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 13:51:58.459945 (+ 104us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 13:51:58.459995 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0924 13:51:58.460082 (+ 87us) server_negotiation.cc:299] Negotiation successful
0924 13:51:58.460099 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76,"thread_start_us":53,"threads_started":1}
I20260924 13:51:58.464844 4114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:58.462232 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51070 (local address 127.13.149.252:37231)
0924 13:51:58.462270 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0924 13:51:58.462272 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:51:58.462300 (+ 28us) server_negotiation.cc:407] Connection header received
0924 13:51:58.462358 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:51:58.462359 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:51:58.462369 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:51:58.462416 (+ 47us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 13:51:58.462554 (+ 138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:51:58.463028 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:51:58.464060 (+ 1032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:51:58.464341 (+ 281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:51:58.464372 (+ 31us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:51:58.464582 (+ 210us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 13:51:58.464618 (+ 36us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 13:51:58.464664 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0924 13:51:58.464739 (+ 75us) server_negotiation.cc:299] Negotiation successful
0924 13:51:58.464763 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260924 13:51:58.464977 4108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:58.461938 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44866 (local address 127.13.149.253:36629)
0924 13:51:58.461980 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0924 13:51:58.461982 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:51:58.462089 (+ 107us) server_negotiation.cc:407] Connection header received
0924 13:51:58.462158 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:51:58.462159 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:51:58.462170 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:51:58.462222 (+ 52us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 13:51:58.462429 (+ 207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:51:58.463231 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:51:58.464174 (+ 943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:51:58.464474 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:51:58.464527 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:51:58.464661 (+ 134us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 13:51:58.464707 (+ 46us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 13:51:58.464749 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0924 13:51:58.464843 (+ 94us) server_negotiation.cc:299] Negotiation successful
0924 13:51:58.464870 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260924 13:51:58.465066 4121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:58.461944 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45820 (local address 127.13.149.254:40929)
0924 13:51:58.462048 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0924 13:51:58.462050 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:51:58.462167 (+ 117us) server_negotiation.cc:407] Connection header received
0924 13:51:58.462279 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:51:58.462280 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:51:58.462337 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:51:58.462413 (+ 76us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 13:51:58.462577 (+ 164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:51:58.463326 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:51:58.464060 (+ 734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:51:58.464364 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:51:58.464422 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:51:58.464612 (+ 190us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 13:51:58.464737 (+ 125us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 13:51:58.464803 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0924 13:51:58.464968 (+ 165us) server_negotiation.cc:299] Negotiation successful
0924 13:51:58.464993 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83,"thread_start_us":54,"threads_started":1}
I20260924 13:51:58.466619 13911 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJN9WRF/build/release/bin/kudu with pid 3756
I20260924 13:51:58.471236 13911 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJN9WRF/build/release/bin/kudu with pid 3831
I20260924 13:51:58.475574 13911 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJN9WRF/build/release/bin/kudu with pid 3906
2026-09-24T13:51:58Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (2718 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790257629041645-13911-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 13:51:58 dist-test-slave-sjl2 krb5kdc[4125](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 13:51:58 dist-test-slave-sjl2 krb5kdc[4125](info): set up 2 sockets
Sep 24 13:51:58 dist-test-slave-sjl2 krb5kdc[4125](info): commencing operation
krb5kdc: starting...
W20260924 13:51:59.438396 13911 mini_kdc.cc:121] Time spent starting KDC: real 0.944s user 0.001s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790257629041645-13911-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790257629041645-13911-0/krb5kdc/test-user.keytab.
Sep 24 13:51:59 dist-test-slave-sjl2 krb5kdc[4125](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257919, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T13:51:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T13:51:59Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.149.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.149.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.149.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.254.keytab.
Entry for principal kudu/127.13.149.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.254.keytab.
I20260924 13:51:59.560108 13911 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJN9WRF/build/release/bin/kudu
/tmp/dist-test-taskJN9WRF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790257629041645-13911-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790257629041645-13911-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790257629041645-13911-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790257629041645-13911-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.149.254:40347
--webserver_interface=127.13.149.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.254.keytab
--principal=kudu/127.13.149.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.149.212:35589
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.149.254:40347,127.13.149.253:43403,127.13.149.252:35377
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790257629041645-13911-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790257629041645-13911-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790257629041645-13911-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 13:51:59.640517 4138 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260924 13:51:59.640753 4138 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 13:51:59.640770 4138 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 13:51:59.640794 4138 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 13:51:59.642114 4138 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 13:51:59.642168 4138 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 13:51:59.642181 4138 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 13:51:59.642194 4138 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 13:51:59.642205 4138 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 13:51:59.643777 4138 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.149.212:35589
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790257629041645-13911-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790257629041645-13911-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.149.254:40347,127.13.149.253:43403,127.13.149.252:35377
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.254.keytab
--principal=kudu/127.13.149.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.149.254:40347
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790257629041645-13911-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.149.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4138
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790257629041645-13911-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 12:29:37 UTC on ccycloud.quasar-iqzbxn.root.comops.site
build id 4670106
I20260924 13:51:59.644013 4138 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 13:51:59.644248 4138 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260924 13:51:59.649410 4138 server_base.cc:1034] running on GCE node
W20260924 13:51:59.649360 4146 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 13:51:59.649385 4143 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 13:51:59.649360 4144 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 13:51:59.649859 4138 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 13:51:59.650173 4138 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 13:51:59.651361 4138 hybrid_clock.cc:648] HybridClock initialized: now 1790257919651348 us; error 32 us; skew 500 ppm
Sep 24 13:51:59 dist-test-slave-sjl2 krb5kdc[4125](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257919, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.149.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 13:51:59.659314 4138 init.cc:377] Logged in from keytab as kudu/127.13.149.254@KRBTEST.COM (short username kudu)
I20260924 13:51:59.659830 4138 webserver.cc:460] Webserver started at http://127.13.149.254:43595/ using document root <none> and password file <none>
I20260924 13:51:59.660091 4138 fs_manager.cc:359] Metadata directory not provided
I20260924 13:51:59.660143 4138 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 13:51:59.660284 4138 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 13:51:59.661427 4138 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790257629041645-13911-0/minicluster-data/master-0/data/instance:
uuid: "e7e47f0da9d5438bb9f6c6a6914aa0d7"
format_stamp: "Formatted at 2026-09-24 13:51:59 on dist-test-slave-sjl2"
server_key: "91aaff8742fc7a19af395e88bedcaa4b"
server_key_iv: "7f4e8227b7a6997b3f4fbc50dfc1ecab"
server_key_version: "encryptionkey@0"
I20260924 13:51:59.661765 4138 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790257629041645-13911-0/minicluster-data/master-0/wal/instance:
uuid: "e7e47f0da9d5438bb9f6c6a6914aa0d7"
format_stamp: "Formatted at 2026-09-24 13:51:59 on dist-test-slave-sjl2"
server_key: "91aaff8742fc7a19af395e88bedcaa4b"
server_key_iv: "7f4e8227b7a6997b3f4fbc50dfc1ecab"
server_key_version: "encryptionkey@0"
I20260924 13:51:59.663064 4138 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260924 13:51:59.663789 4153 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 13:51:59.663930 4138 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 13:51:59.664018 4138 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790257629041645-13911-0/minicluster-data/master-0/data,/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790257629041645-13911-0/minicluster-data/master-0/wal
uuid: "e7e47f0da9d5438bb9f6c6a6914aa0d7"
format_stamp: "Formatted at 2026-09-24 13:51:59 on dist-test-slave-sjl2"
server_key: "91aaff8742fc7a19af395e88bedcaa4b"
server_key_iv: "7f4e8227b7a6997b3f4fbc50dfc1ecab"
server_key_version: "encryptionkey@0"
I20260924 13:51:59.664075 4138 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790257629041645-13911-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790257629041645-13911-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790257629041645-13911-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 13:51:59.672263 4138 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 13:51:59.674364 4138 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 13:51:59.674501 4138 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 13:51:59.678372 4138 rpc_server.cc:307] RPC server started. Bound to: 127.13.149.254:40347
I20260924 13:51:59.678473 4205 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.149.254:40347 every 8 connection(s)
I20260924 13:51:59.679001 4138 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790257629041645-13911-0/minicluster-data/master-0/data/info.pb
I20260924 13:51:59.680006 4206 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 13:51:59.681658 4206 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.149.254" port: 40347 } has no permanent_uuid. Determining permanent_uuid...
I20260924 13:51:59.684993 13911 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJN9WRF/build/release/bin/kudu as pid 4138
I20260924 13:51:59.685079 13911 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790257629041645-13911-0/minicluster-data/master-0/wal/instance
I20260924 13:51:59.685248 13911 external_mini_cluster.cc:1428] Setting key bb80d5ad68d65033851374a294f68061
Sep 24 13:51:59 dist-test-slave-sjl2 krb5kdc[4125](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257919, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.149.254@KRBTEST.COM for kudu/127.13.149.254@KRBTEST.COM
I20260924 13:51:59.691304 4209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:59.682401 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33892 (local address 127.13.149.254:40347)
0924 13:51:59.682545 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0924 13:51:59.682554 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0924 13:51:59.682675 (+ 121us) server_negotiation.cc:407] Connection header received
0924 13:51:59.683266 (+ 591us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:51:59.683268 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:51:59.683285 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:51:59.683361 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:51:59.683923 (+ 562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:51:59.684773 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:51:59.685786 (+ 1013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:51:59.686089 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:51:59.686182 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:51:59.688333 (+ 2151us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:51:59.688340 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:51:59.688341 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 13:51:59.688348 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:51:59.689855 (+ 1507us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:51:59.690192 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:51:59.690195 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:51:59.690195 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:51:59.690280 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:51:59.690470 (+ 190us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:51:59.690471 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:51:59.690471 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:51:59.690685 (+ 214us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:51:59.690764 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0924 13:51:59.690937 (+ 173us) server_negotiation.cc:299] Negotiation successful
0924 13:51:59.691045 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":99,"thread_start_us":62,"threads_started":1}
I20260924 13:51:59.691304 4208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:59.682351 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:40347 (local address 127.0.0.1:33892)
0924 13:51:59.682536 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0924 13:51:59.682583 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0924 13:51:59.683109 (+ 526us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:51:59.683371 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:51:59.683373 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:51:59.683657 (+ 284us) client_negotiation.cc:189] Negotiated authn=SASL
0924 13:51:59.683822 (+ 165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:51:59.683827 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:51:59.684863 (+ 1036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:51:59.684865 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:51:59.685703 (+ 838us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:51:59.685705 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:51:59.686162 (+ 457us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:51:59.686163 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:51:59.686229 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:51:59.686745 (+ 516us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 13:51:59.686754 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0924 13:51:59.688246 (+ 1492us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 13:51:59.689931 (+ 1685us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 13:51:59.689937 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 13:51:59.689940 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0924 13:51:59.690125 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 13:51:59.690335 (+ 210us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 13:51:59.690336 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 13:51:59.690336 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 13:51:59.690418 (+ 82us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 13:51:59.690750 (+ 332us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 13:51:59.690755 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 13:51:59.690818 (+ 63us) client_negotiation.cc:769] Sending connection context
0924 13:51:59.690903 (+ 85us) client_negotiation.cc:240] Negotiation successful
0924 13:51:59.691119 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"mutex_wait_us":75,"thread_start_us":89,"threads_started":1}
WARNING: no policy specified for kudu/127.13.149.253@KRBTEST.COM; defaulting to no policy
I20260924 13:51:59.692039 4206 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.149.253" port: 43403 } has no permanent_uuid. Determining permanent_uuid...
I20260924 13:51:59.692402 4208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:59.692285 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.253:43403 (local address 127.0.0.1:36886)
0924 13:51:59.692334 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0924 13:51:59.692379 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.149.253:43403: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260924 13:51:59.692530 4157 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.13.149.253:43403: connect: Connection refused (error 111)
I20260924 13:51:59.692984 4208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:59.692878 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.253:43403 (local address 127.0.0.1:36892)
0924 13:51:59.692940 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0924 13:51:59.692969 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.149.253:43403: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
W20260924 13:51:59.693073 4206 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.149.253:43403: Network error: Client connection negotiation failed: client connection to 127.13.149.253:43403: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.149.253@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.149.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.253.keytab.
Entry for principal kudu/127.13.149.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.253.keytab.
I20260924 13:51:59.712762 13911 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJN9WRF/build/release/bin/kudu
/tmp/dist-test-taskJN9WRF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790257629041645-13911-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790257629041645-13911-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790257629041645-13911-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790257629041645-13911-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.149.253:43403
--webserver_interface=127.13.149.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.253.keytab
--principal=kudu/127.13.149.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.149.212:35589
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.149.254:40347,127.13.149.253:43403,127.13.149.252:35377
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790257629041645-13911-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790257629041645-13911-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790257629041645-13911-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260924 13:51:59.742309 4206 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.149.253" port: 43403 } attempt: 1
I20260924 13:51:59.742810 4208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:59.742686 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.253:43403 (local address 127.0.0.1:36896)
0924 13:51:59.742742 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0924 13:51:59.742777 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.149.253:43403: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
I20260924 13:51:59.743165 4208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:59.743041 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.253:43403 (local address 127.0.0.1:36912)
0924 13:51:59.743078 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0924 13:51:59.743101 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.149.253:43403: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
W20260924 13:51:59.743247 4206 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.149.253:43403: Network error: Client connection negotiation failed: client connection to 127.13.149.253:43403: connect: Connection refused (error 111)
W20260924 13:51:59.794174 4213 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260924 13:51:59.794422 4213 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 13:51:59.794448 4213 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 13:51:59.794476 4213 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 13:51:59.795636 4213 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 13:51:59.795675 4213 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 13:51:59.795692 4213 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 13:51:59.795708 4213 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 13:51:59.795724 4213 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 13:51:59.797214 4213 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.149.212:35589
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790257629041645-13911-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790257629041645-13911-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.13.149.254:40347,127.13.149.253:43403,127.13.149.252:35377
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.253.keytab
--principal=kudu/127.13.149.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.149.253:43403
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790257629041645-13911-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.13.149.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4213
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790257629041645-13911-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 12:29:37 UTC on ccycloud.quasar-iqzbxn.root.comops.site
build id 4670106
I20260924 13:51:59.797456 4213 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 13:51:59.797662 4213 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 13:51:59.802258 4221 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 13:51:59.802269 4219 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 13:51:59.802258 4218 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 13:51:59.802485 4213 server_base.cc:1034] running on GCE node
I20260924 13:51:59.802733 4213 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 13:51:59.803020 4213 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 13:51:59.804157 4213 hybrid_clock.cc:648] HybridClock initialized: now 1790257919804142 us; error 26 us; skew 500 ppm
Sep 24 13:51:59 dist-test-slave-sjl2 krb5kdc[4125](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257919, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.149.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 13:51:59.806677 4213 init.cc:377] Logged in from keytab as kudu/127.13.149.253@KRBTEST.COM (short username kudu)
I20260924 13:51:59.807101 4213 webserver.cc:460] Webserver started at http://127.13.149.253:33525/ using document root <none> and password file <none>
I20260924 13:51:59.807317 4213 fs_manager.cc:359] Metadata directory not provided
I20260924 13:51:59.807359 4213 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 13:51:59.807475 4213 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 13:51:59.808544 4213 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790257629041645-13911-0/minicluster-data/master-1/data/instance:
uuid: "4e0a602154dc454ab7defcc1ec52df99"
format_stamp: "Formatted at 2026-09-24 13:51:59 on dist-test-slave-sjl2"
server_key: "813b5034543fc08af1cb70fcbec54ff6"
server_key_iv: "7dd61e3858c7e863807cfec5a6094d3f"
server_key_version: "encryptionkey@0"
I20260924 13:51:59.808861 4213 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790257629041645-13911-0/minicluster-data/master-1/wal/instance:
uuid: "4e0a602154dc454ab7defcc1ec52df99"
format_stamp: "Formatted at 2026-09-24 13:51:59 on dist-test-slave-sjl2"
server_key: "813b5034543fc08af1cb70fcbec54ff6"
server_key_iv: "7dd61e3858c7e863807cfec5a6094d3f"
server_key_version: "encryptionkey@0"
I20260924 13:51:59.810353 4213 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260924 13:51:59.811034 4228 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 13:51:59.811196 4213 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260924 13:51:59.811275 4213 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790257629041645-13911-0/minicluster-data/master-1/data,/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790257629041645-13911-0/minicluster-data/master-1/wal
uuid: "4e0a602154dc454ab7defcc1ec52df99"
format_stamp: "Formatted at 2026-09-24 13:51:59 on dist-test-slave-sjl2"
server_key: "813b5034543fc08af1cb70fcbec54ff6"
server_key_iv: "7dd61e3858c7e863807cfec5a6094d3f"
server_key_version: "encryptionkey@0"
I20260924 13:51:59.811342 4213 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790257629041645-13911-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790257629041645-13911-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790257629041645-13911-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 13:51:59.811383 4206 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.149.253" port: 43403 } attempt: 2
I20260924 13:51:59.811746 4208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:59.811625 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.253:43403 (local address 127.0.0.1:36926)
0924 13:51:59.811682 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0924 13:51:59.811712 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.149.253:43403: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260924 13:51:59.812068 4208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:59.811989 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.253:43403 (local address 127.0.0.1:36930)
0924 13:51:59.812024 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0924 13:51:59.812047 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.149.253:43403: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
W20260924 13:51:59.812152 4206 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.149.253:43403: Network error: Client connection negotiation failed: client connection to 127.13.149.253:43403: connect: Connection refused (error 111)
I20260924 13:51:59.829846 4213 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 13:51:59.832473 4213 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 13:51:59.832616 4213 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 13:51:59.836380 4213 rpc_server.cc:307] RPC server started. Bound to: 127.13.149.253:43403
I20260924 13:51:59.836424 4280 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.149.253:43403 every 8 connection(s)
I20260924 13:51:59.836885 4213 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790257629041645-13911-0/minicluster-data/master-1/data/info.pb
I20260924 13:51:59.837633 13911 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJN9WRF/build/release/bin/kudu as pid 4213
I20260924 13:51:59.837729 13911 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790257629041645-13911-0/minicluster-data/master-1/wal/instance
I20260924 13:51:59.837929 13911 external_mini_cluster.cc:1428] Setting key ab117a1e7e15eaa0dbe15ad694ef65dc
I20260924 13:51:59.837930 4281 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 13:51:59.839473 4281 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.149.254" port: 40347 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.13.149.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.149.252@KRBTEST.COM" created.
Sep 24 13:51:59 dist-test-slave-sjl2 krb5kdc[4125](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257919, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.149.253@KRBTEST.COM for kudu/127.13.149.254@KRBTEST.COM
I20260924 13:51:59.852763 4209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:59.840197 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33894 (local address 127.13.149.254:40347)
0924 13:51:59.840262 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0924 13:51:59.840264 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:51:59.840521 (+ 257us) server_negotiation.cc:407] Connection header received
0924 13:51:59.841230 (+ 709us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:51:59.841232 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:51:59.841249 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:51:59.841322 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:51:59.842199 (+ 877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:51:59.842694 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:51:59.844206 (+ 1512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:51:59.844415 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:51:59.844548 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:51:59.849809 (+ 5261us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:51:59.849811 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:51:59.849812 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 13:51:59.849822 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:51:59.851327 (+ 1505us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:51:59.851806 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:51:59.851807 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:51:59.851808 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:51:59.851866 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:51:59.852100 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:51:59.852101 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:51:59.852101 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:51:59.852219 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:51:59.852281 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0924 13:51:59.852566 (+ 285us) server_negotiation.cc:299] Negotiation successful
0924 13:51:59.852656 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260924 13:51:59.852969 4284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:59.840176 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:40347 (local address 127.0.0.1:33894)
0924 13:51:59.840360 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0924 13:51:59.840411 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0924 13:51:59.841092 (+ 681us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:51:59.841403 (+ 311us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:51:59.841408 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:51:59.841777 (+ 369us) client_negotiation.cc:189] Negotiated authn=SASL
0924 13:51:59.842080 (+ 303us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:51:59.842088 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:51:59.842795 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:51:59.842796 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:51:59.844099 (+ 1303us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:51:59.844103 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:51:59.844575 (+ 472us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:51:59.844583 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:51:59.844837 (+ 254us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:51:59.845514 (+ 677us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 13:51:59.845525 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0924 13:51:59.849675 (+ 4150us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 13:51:59.851429 (+ 1754us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 13:51:59.851440 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 13:51:59.851442 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 13:51:59.851719 (+ 277us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 13:51:59.851925 (+ 206us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 13:51:59.851926 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 13:51:59.851926 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 13:51:59.852041 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 13:51:59.852300 (+ 259us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 13:51:59.852308 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 13:51:59.852448 (+ 140us) client_negotiation.cc:769] Sending connection context
0924 13:51:59.852538 (+ 90us) client_negotiation.cc:240] Negotiation successful
0924 13:51:59.852712 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":98,"threads_started":1}
I20260924 13:51:59.853525 4281 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.149.253" port: 43403 } has no permanent_uuid. Determining permanent_uuid...
Sep 24 13:51:59 dist-test-slave-sjl2 krb5kdc[4125](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257919, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.149.253@KRBTEST.COM for kudu/127.13.149.253@KRBTEST.COM
I20260924 13:51:59.861689 4284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:59.853794 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.253:43403 (local address 127.0.0.1:36944)
0924 13:51:59.853836 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0924 13:51:59.853990 (+ 154us) client_negotiation.cc:174] Beginning negotiation
0924 13:51:59.854034 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:51:59.854175 (+ 141us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:51:59.854176 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:51:59.854319 (+ 143us) client_negotiation.cc:189] Negotiated authn=SASL
0924 13:51:59.854374 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:51:59.854376 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:51:59.855203 (+ 827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:51:59.855204 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:51:59.855707 (+ 503us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:51:59.855708 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:51:59.856163 (+ 455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:51:59.856164 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:51:59.856237 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:51:59.856600 (+ 363us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 13:51:59.856604 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0924 13:51:59.859114 (+ 2510us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 13:51:59.860695 (+ 1581us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 13:51:59.860696 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 13:51:59.860697 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 13:51:59.860859 (+ 162us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 13:51:59.861057 (+ 198us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 13:51:59.861058 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 13:51:59.861059 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 13:51:59.861111 (+ 52us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 13:51:59.861379 (+ 268us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 13:51:59.861379 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 13:51:59.861417 (+ 38us) client_negotiation.cc:769] Sending connection context
0924 13:51:59.861476 (+ 59us) client_negotiation.cc:240] Negotiation successful
0924 13:51:59.861578 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":24}
I20260924 13:51:59.861771 4286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:59.853870 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36944 (local address 127.13.149.253:43403)
0924 13:51:59.854018 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0924 13:51:59.854023 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 13:51:59.854043 (+ 20us) server_negotiation.cc:407] Connection header received
0924 13:51:59.854085 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:51:59.854088 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:51:59.854101 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:51:59.854171 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:51:59.854437 (+ 266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:51:59.855127 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:51:59.855774 (+ 647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:51:59.856101 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:51:59.856161 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:51:59.859198 (+ 3037us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:51:59.859209 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:51:59.859209 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 13:51:59.859217 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:51:59.860617 (+ 1400us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:51:59.860949 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:51:59.860952 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:51:59.860953 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:51:59.861006 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:51:59.861159 (+ 153us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:51:59.861160 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:51:59.861160 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:51:59.861322 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:51:59.861391 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0924 13:51:59.861504 (+ 113us) server_negotiation.cc:299] Negotiation successful
0924 13:51:59.861630 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":62,"server-negotiator.queue_time_us":119,"thread_start_us":76,"threads_started":1}
I20260924 13:51:59.862460 4281 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.149.252" port: 35377 } has no permanent_uuid. Determining permanent_uuid...
I20260924 13:51:59.862875 4284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:59.862682 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.252:35377 (local address 127.0.0.1:34236)
0924 13:51:59.862746 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0924 13:51:59.862830 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.149.252:35377: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260924 13:51:59.863018 4232 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.13.149.252:35377: connect: Connection refused (error 111)
I20260924 13:51:59.863476 4284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:59.863376 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.252:35377 (local address 127.0.0.1:34250)
0924 13:51:59.863403 (+ 27us) negotiation.cc:107] Waiting for socket to connect
0924 13:51:59.863432 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.149.252:35377: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":14}
W20260924 13:51:59.863564 4281 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.149.252:35377: Network error: Client connection negotiation failed: client connection to 127.13.149.252:35377: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.149.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.252.keytab.
Entry for principal kudu/127.13.149.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.252.keytab.
I20260924 13:51:59.864603 13911 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJN9WRF/build/release/bin/kudu
/tmp/dist-test-taskJN9WRF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790257629041645-13911-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790257629041645-13911-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790257629041645-13911-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790257629041645-13911-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.149.252:35377
--webserver_interface=127.13.149.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.252.keytab
--principal=kudu/127.13.149.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.149.212:35589
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.149.254:40347,127.13.149.253:43403,127.13.149.252:35377
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790257629041645-13911-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790257629041645-13911-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790257629041645-13911-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260924 13:51:59.903345 4206 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.149.253" port: 43403 } attempt: 3
Sep 24 13:51:59 dist-test-slave-sjl2 krb5kdc[4125](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257919, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.149.254@KRBTEST.COM for kudu/127.13.149.253@KRBTEST.COM
I20260924 13:51:59.910782 4286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:59.903881 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36956 (local address 127.13.149.253:43403)
0924 13:51:59.903954 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0924 13:51:59.903956 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:51:59.903964 (+ 8us) server_negotiation.cc:407] Connection header received
0924 13:51:59.903990 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:51:59.903990 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:51:59.904000 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:51:59.904044 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:51:59.904379 (+ 335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:51:59.904880 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:51:59.905626 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:51:59.905851 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:51:59.905909 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:51:59.908124 (+ 2215us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:51:59.908126 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:51:59.908126 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 13:51:59.908136 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:51:59.909614 (+ 1478us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:51:59.910005 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:51:59.910007 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:51:59.910007 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:51:59.910069 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:51:59.910251 (+ 182us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:51:59.910252 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:51:59.910252 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:51:59.910378 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:51:59.910442 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0924 13:51:59.910595 (+ 153us) server_negotiation.cc:299] Negotiation successful
0924 13:51:59.910675 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260924 13:51:59.910784 4208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:59.903779 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.253:43403 (local address 127.0.0.1:36956)
0924 13:51:59.903847 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0924 13:51:59.903858 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0924 13:51:59.903900 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:51:59.904058 (+ 158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:51:59.904059 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:51:59.904224 (+ 165us) client_negotiation.cc:189] Negotiated authn=SASL
0924 13:51:59.904294 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:51:59.904296 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:51:59.904967 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:51:59.904969 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:51:59.905534 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:51:59.905537 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:51:59.905914 (+ 377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:51:59.905915 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:51:59.905986 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:51:59.906515 (+ 529us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 13:51:59.906521 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0924 13:51:59.908011 (+ 1490us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 13:51:59.909718 (+ 1707us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 13:51:59.909719 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 13:51:59.909720 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 13:51:59.909928 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 13:51:59.910133 (+ 205us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 13:51:59.910133 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 13:51:59.910134 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 13:51:59.910195 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 13:51:59.910440 (+ 245us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 13:51:59.910441 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 13:51:59.910492 (+ 51us) client_negotiation.cc:769] Sending connection context
0924 13:51:59.910559 (+ 67us) client_negotiation.cc:240] Negotiation successful
0924 13:51:59.910670 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42}
I20260924 13:51:59.911405 4206 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.149.252" port: 35377 } has no permanent_uuid. Determining permanent_uuid...
I20260924 13:51:59.911804 4208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:59.911697 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.252:35377 (local address 127.0.0.1:34262)
0924 13:51:59.911742 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0924 13:51:59.911775 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.149.252:35377: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
I20260924 13:51:59.912194 4208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:59.912109 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.252:35377 (local address 127.0.0.1:34266)
0924 13:51:59.912148 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0924 13:51:59.912174 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.149.252:35377: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260924 13:51:59.912281 4206 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.149.252:35377: Network error: Client connection negotiation failed: client connection to 127.13.149.252:35377: connect: Connection refused (error 111)
I20260924 13:51:59.912770 4281 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.149.252" port: 35377 } attempt: 1
I20260924 13:51:59.913187 4284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:59.913035 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.252:35377 (local address 127.0.0.1:34276)
0924 13:51:59.913127 (+ 92us) negotiation.cc:107] Waiting for socket to connect
0924 13:51:59.913161 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.149.252:35377: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260924 13:51:59.913512 4284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:59.913422 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.252:35377 (local address 127.0.0.1:34280)
0924 13:51:59.913461 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0924 13:51:59.913486 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.149.252:35377: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
W20260924 13:51:59.913600 4281 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.149.252:35377: Network error: Client connection negotiation failed: client connection to 127.13.149.252:35377: connect: Connection refused (error 111)
I20260924 13:51:59.943475 4206 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.149.252" port: 35377 } attempt: 1
I20260924 13:51:59.943996 4208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:59.943839 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.252:35377 (local address 127.0.0.1:34286)
0924 13:51:59.943909 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0924 13:51:59.943959 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.149.252:35377: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260924 13:51:59.944339 4208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:59.944243 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.252:35377 (local address 127.0.0.1:34288)
0924 13:51:59.944289 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0924 13:51:59.944318 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.149.252:35377: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260924 13:51:59.944437 4206 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.149.252:35377: Network error: Client connection negotiation failed: client connection to 127.13.149.252:35377: connect: Connection refused (error 111)
W20260924 13:51:59.949680 4288 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260924 13:51:59.949913 4288 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 13:51:59.949931 4288 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 13:51:59.949954 4288 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 13:51:59.951244 4288 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 13:51:59.951298 4288 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 13:51:59.951311 4288 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 13:51:59.951323 4288 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 13:51:59.951334 4288 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 13:51:59.952981 4288 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.149.212:35589
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790257629041645-13911-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790257629041645-13911-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.13.149.254:40347,127.13.149.253:43403,127.13.149.252:35377
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790257629041645-13911-0/krb5kdc/kudu_127.13.149.252.keytab
--principal=kudu/127.13.149.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.149.252:35377
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790257629041645-13911-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.13.149.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4288
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790257629041645-13911-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 12:29:37 UTC on ccycloud.quasar-iqzbxn.root.comops.site
build id 4670106
I20260924 13:51:59.953248 4288 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 13:51:59.953473 4288 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 13:51:59.958614 4293 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 13:51:59.958810 4288 server_base.cc:1034] running on GCE node
W20260924 13:51:59.958643 4294 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 13:51:59.958945 4296 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 13:51:59.959203 4288 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 13:51:59.959479 4288 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 13:51:59.960666 4288 hybrid_clock.cc:648] HybridClock initialized: now 1790257919960644 us; error 38 us; skew 500 ppm
Sep 24 13:51:59 dist-test-slave-sjl2 krb5kdc[4125](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257919, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.149.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 13:51:59.963466 4288 init.cc:377] Logged in from keytab as kudu/127.13.149.252@KRBTEST.COM (short username kudu)
I20260924 13:51:59.963919 4288 webserver.cc:460] Webserver started at http://127.13.149.252:34653/ using document root <none> and password file <none>
I20260924 13:51:59.964150 4288 fs_manager.cc:359] Metadata directory not provided
I20260924 13:51:59.964198 4288 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 13:51:59.964337 4288 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 13:51:59.965489 4288 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790257629041645-13911-0/minicluster-data/master-2/data/instance:
uuid: "41d8e6ee66994e6b90a42b5de18274ab"
format_stamp: "Formatted at 2026-09-24 13:51:59 on dist-test-slave-sjl2"
server_key: "a8390651b60133d7ded266b858491ff7"
server_key_iv: "a1bf32065680ba63eb9aa7fc412b9e04"
server_key_version: "encryptionkey@0"
I20260924 13:51:59.965817 4288 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790257629041645-13911-0/minicluster-data/master-2/wal/instance:
uuid: "41d8e6ee66994e6b90a42b5de18274ab"
format_stamp: "Formatted at 2026-09-24 13:51:59 on dist-test-slave-sjl2"
server_key: "a8390651b60133d7ded266b858491ff7"
server_key_iv: "a1bf32065680ba63eb9aa7fc412b9e04"
server_key_version: "encryptionkey@0"
I20260924 13:51:59.967201 4288 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260924 13:51:59.968006 4303 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 13:51:59.968226 4288 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260924 13:51:59.968313 4288 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790257629041645-13911-0/minicluster-data/master-2/data,/tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790257629041645-13911-0/minicluster-data/master-2/wal
uuid: "41d8e6ee66994e6b90a42b5de18274ab"
format_stamp: "Formatted at 2026-09-24 13:51:59 on dist-test-slave-sjl2"
server_key: "a8390651b60133d7ded266b858491ff7"
server_key_iv: "a1bf32065680ba63eb9aa7fc412b9e04"
server_key_version: "encryptionkey@0"
I20260924 13:51:59.968391 4288 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790257629041645-13911-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790257629041645-13911-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790257629041645-13911-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 13:51:59.979481 4288 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 13:51:59.981732 4288 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 13:51:59.981736 4281 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.149.252" port: 35377 } attempt: 2
I20260924 13:51:59.981882 4288 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 13:51:59.982126 4284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:59.981981 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.252:35377 (local address 127.0.0.1:34304)
0924 13:51:59.982048 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0924 13:51:59.982089 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.149.252:35377: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260924 13:51:59.982517 4284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:59.982424 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.252:35377 (local address 127.0.0.1:34316)
0924 13:51:59.982466 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0924 13:51:59.982494 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.149.252:35377: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260924 13:51:59.982609 4281 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.149.252:35377: Network error: Client connection negotiation failed: client connection to 127.13.149.252:35377: connect: Connection refused (error 111)
I20260924 13:51:59.985775 4288 rpc_server.cc:307] RPC server started. Bound to: 127.13.149.252:35377
I20260924 13:51:59.985806 4355 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.149.252:35377 every 8 connection(s)
I20260924 13:51:59.986140 4288 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790257629041645-13911-0/minicluster-data/master-2/data/info.pb
I20260924 13:51:59.986966 4356 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 13:51:59.988447 4356 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.149.254" port: 40347 } has no permanent_uuid. Determining permanent_uuid...
I20260924 13:51:59.989437 13911 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJN9WRF/build/release/bin/kudu as pid 4288
I20260924 13:51:59.989552 13911 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJN9WRF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790257629041645-13911-0/minicluster-data/master-2/wal/instance
I20260924 13:51:59.989755 13911 external_mini_cluster.cc:1428] Setting key 82132c7b9c2b19fdf4f84c92726335dd
Sep 24 13:51:59 dist-test-slave-sjl2 krb5kdc[4125](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257919, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.149.252@KRBTEST.COM for kudu/127.13.149.254@KRBTEST.COM
Sep 24 13:51:59 dist-test-slave-sjl2 krb5kdc[4125](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257919, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.149.252@KRBTEST.COM
I20260924 13:51:59.998176 4209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:59.989184 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33900 (local address 127.13.149.254:40347)
0924 13:51:59.989247 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0924 13:51:59.989248 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0924 13:51:59.989473 (+ 225us) server_negotiation.cc:407] Connection header received
0924 13:51:59.990040 (+ 567us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:51:59.990042 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:51:59.990055 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:51:59.990115 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:51:59.990895 (+ 780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:51:59.991380 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:51:59.992557 (+ 1177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:51:59.992787 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:51:59.992844 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:51:59.994930 (+ 2086us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:51:59.994932 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:51:59.994932 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 13:51:59.994940 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:51:59.996557 (+ 1617us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:51:59.996969 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:51:59.996971 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:51:59.996971 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:51:59.997034 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:51:59.997298 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:51:59.997298 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:51:59.997299 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:51:59.997425 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:51:59.997495 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0924 13:51:59.997983 (+ 488us) server_negotiation.cc:299] Negotiation successful
0924 13:51:59.998077 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260924 13:51:59.998421 4358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:59.989174 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.254:40347 (local address 127.0.0.1:33900)
0924 13:51:59.989360 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0924 13:51:59.989395 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0924 13:51:59.989937 (+ 542us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:51:59.990191 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:51:59.990195 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:51:59.990541 (+ 346us) client_negotiation.cc:189] Negotiated authn=SASL
0924 13:51:59.990814 (+ 273us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:51:59.990820 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:51:59.991454 (+ 634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:51:59.991455 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:51:59.992464 (+ 1009us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:51:59.992467 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:51:59.992857 (+ 390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:51:59.992858 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:51:59.993011 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:51:59.993485 (+ 474us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 13:51:59.993493 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0924 13:51:59.994846 (+ 1353us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 13:51:59.996642 (+ 1796us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 13:51:59.996648 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 13:51:59.996650 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 13:51:59.996878 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 13:51:59.997098 (+ 220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 13:51:59.997098 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 13:51:59.997099 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 13:51:59.997234 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 13:51:59.997520 (+ 286us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 13:51:59.997526 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 13:51:59.997700 (+ 174us) client_negotiation.cc:769] Sending connection context
0924 13:51:59.997960 (+ 260us) client_negotiation.cc:240] Negotiation successful
0924 13:51:59.998137 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"mutex_wait_us":205,"thread_start_us":111,"threads_started":1}
I20260924 13:51:59.998998 4356 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.149.253" port: 43403 } has no permanent_uuid. Determining permanent_uuid...
I20260924 13:51:59.999481 4359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:59.990082 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34320 (local address 127.13.149.252:35377)
0924 13:51:59.990227 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0924 13:51:59.990231 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 13:51:59.990405 (+ 174us) server_negotiation.cc:407] Connection header received
0924 13:51:59.990467 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:51:59.990469 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:51:59.990487 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:51:59.990550 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:51:59.991199 (+ 649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:51:59.992004 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:51:59.992691 (+ 687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:51:59.993031 (+ 340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:51:59.993114 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:51:59.995387 (+ 2273us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:51:59.995397 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:51:59.995397 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 13:51:59.995404 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:51:59.997083 (+ 1679us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:51:59.997435 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:51:59.997438 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:51:59.997438 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:51:59.997494 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:51:59.998057 (+ 563us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:51:59.998058 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:51:59.998059 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:51:59.998247 (+ 188us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:51:59.998452 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0924 13:51:59.999272 (+ 820us) server_negotiation.cc:299] Negotiation successful
0924 13:51:59.999368 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":111,"thread_start_us":79,"threads_started":1}
Sep 24 13:51:59 dist-test-slave-sjl2 krb5kdc[4125](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257919, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.149.252@KRBTEST.COM for kudu/127.13.149.253@KRBTEST.COM
Sep 24 13:52:00 dist-test-slave-sjl2 krb5kdc[4125](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257919, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.149.254@KRBTEST.COM
I20260924 13:52:00.008354 4358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:59.999267 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.253:43403 (local address 127.0.0.1:36966)
0924 13:51:59.999318 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0924 13:51:59.999329 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0924 13:51:59.999371 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:51:59.999477 (+ 106us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:51:59.999478 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:51:59.999615 (+ 137us) client_negotiation.cc:189] Negotiated authn=SASL
0924 13:51:59.999678 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:51:59.999680 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:52:00.000581 (+ 901us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:52:00.000582 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:52:00.001306 (+ 724us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:52:00.001309 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:52:00.001956 (+ 647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:52:00.001958 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:52:00.002046 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:52:00.002535 (+ 489us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 13:52:00.002540 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 13:52:00.004284 (+ 1744us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 13:52:00.006824 (+ 2540us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 13:52:00.006825 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 13:52:00.006826 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 13:52:00.007012 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 13:52:00.007212 (+ 200us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 13:52:00.007213 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 13:52:00.007213 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 13:52:00.007274 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 13:52:00.007755 (+ 481us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 13:52:00.007757 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 13:52:00.007897 (+ 140us) client_negotiation.cc:769] Sending connection context
0924 13:52:00.008006 (+ 109us) client_negotiation.cc:240] Negotiation successful
0924 13:52:00.008221 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260924 13:52:00.008354 4286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:51:59.999342 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36966 (local address 127.13.149.253:43403)
0924 13:51:59.999384 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0924 13:51:59.999386 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:51:59.999394 (+ 8us) server_negotiation.cc:407] Connection header received
0924 13:51:59.999419 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:51:59.999419 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:51:59.999430 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:51:59.999472 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:51:59.999940 (+ 468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:52:00.000506 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:52:00.001419 (+ 913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:52:00.001787 (+ 368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:52:00.001864 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:52:00.004404 (+ 2540us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:52:00.004406 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:52:00.004406 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 13:52:00.004415 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:52:00.006721 (+ 2306us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:52:00.007096 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:52:00.007098 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:52:00.007098 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:52:00.007155 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:52:00.007327 (+ 172us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:52:00.007328 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:52:00.007328 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:52:00.007584 (+ 256us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:52:00.007730 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0924 13:52:00.008069 (+ 339us) server_negotiation.cc:299] Negotiation successful
0924 13:52:00.008221 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260924 13:52:00.009202 4356 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.149.252" port: 35377 } has no permanent_uuid. Determining permanent_uuid...
I20260924 13:52:00.010793 4209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:52:00.000440 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33906 (local address 127.13.149.254:40347)
0924 13:52:00.000482 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0924 13:52:00.000484 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:52:00.000539 (+ 55us) server_negotiation.cc:407] Connection header received
0924 13:52:00.000605 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:52:00.000606 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:52:00.000618 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:52:00.000717 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:52:00.001649 (+ 932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:52:00.002517 (+ 868us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:52:00.003269 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:52:00.003502 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:52:00.003563 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:52:00.006829 (+ 3266us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:52:00.006832 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:52:00.006832 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 13:52:00.006842 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:52:00.008357 (+ 1515us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:52:00.009540 (+ 1183us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:52:00.009541 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:52:00.009542 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:52:00.009615 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:52:00.009891 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:52:00.009894 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:52:00.009894 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:52:00.010151 (+ 257us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:52:00.010514 (+ 363us) server_negotiation.cc:1035] Waiting for connection context
0924 13:52:00.010565 (+ 51us) server_negotiation.cc:299] Negotiation successful
0924 13:52:00.010670 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
Sep 24 13:52:00 dist-test-slave-sjl2 krb5kdc[4125](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257919, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.149.252@KRBTEST.COM for kudu/127.13.149.252@KRBTEST.COM
Sep 24 13:52:00 dist-test-slave-sjl2 krb5kdc[4125](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257919, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.149.253@KRBTEST.COM
I20260924 13:52:00.019614 4358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:52:00.009562 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.252:35377 (local address 127.0.0.1:34330)
0924 13:52:00.009650 (+ 88us) negotiation.cc:107] Waiting for socket to connect
0924 13:52:00.009660 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 13:52:00.009942 (+ 282us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:52:00.010097 (+ 155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:52:00.010099 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:52:00.010311 (+ 212us) client_negotiation.cc:189] Negotiated authn=SASL
0924 13:52:00.010422 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:52:00.010424 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:52:00.011298 (+ 874us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:52:00.011299 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:52:00.011966 (+ 667us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:52:00.011968 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:52:00.012546 (+ 578us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:52:00.012548 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:52:00.012623 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:52:00.013367 (+ 744us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 13:52:00.013372 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 13:52:00.014781 (+ 1409us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 13:52:00.016909 (+ 2128us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 13:52:00.016911 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 13:52:00.016911 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 13:52:00.017178 (+ 267us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 13:52:00.018354 (+ 1176us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 13:52:00.018356 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 13:52:00.018356 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 13:52:00.018474 (+ 118us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 13:52:00.019161 (+ 687us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 13:52:00.019162 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 13:52:00.019219 (+ 57us) client_negotiation.cc:769] Sending connection context
0924 13:52:00.019321 (+ 102us) client_negotiation.cc:240] Negotiation successful
0924 13:52:00.019470 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":69}
I20260924 13:52:00.019615 4206 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.149.252" port: 35377 } attempt: 2
I20260924 13:52:00.019699 4359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:52:00.009614 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34330 (local address 127.13.149.252:35377)
0924 13:52:00.009653 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0924 13:52:00.009655 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:52:00.009954 (+ 299us) server_negotiation.cc:407] Connection header received
0924 13:52:00.010004 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:52:00.010005 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:52:00.010018 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:52:00.010064 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:52:00.010527 (+ 463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:52:00.011194 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:52:00.012122 (+ 928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:52:00.012438 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:52:00.012560 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:52:00.014871 (+ 2311us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:52:00.014872 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:52:00.014873 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 13:52:00.014879 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:52:00.016790 (+ 1911us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:52:00.018171 (+ 1381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:52:00.018173 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:52:00.018173 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:52:00.018238 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:52:00.018652 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:52:00.018653 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:52:00.018653 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:52:00.018848 (+ 195us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:52:00.019068 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0924 13:52:00.019359 (+ 291us) server_negotiation.cc:299] Negotiation successful
0924 13:52:00.019552 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":112,"server-negotiator.queue_time_us":18}
I20260924 13:52:00.021708 4356 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 41d8e6ee66994e6b90a42b5de18274ab: Bootstrap starting.
I20260924 13:52:00.022404 4356 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 41d8e6ee66994e6b90a42b5de18274ab: Neither blocks nor log segments found. Creating new log.
I20260924 13:52:00.022753 4356 log.cc:824] T 00000000000000000000000000000000 P 41d8e6ee66994e6b90a42b5de18274ab: Log is configured to *not* fsync() on all Append() calls
I20260924 13:52:00.024573 4356 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 41d8e6ee66994e6b90a42b5de18274ab: No bootstrap required, opened a new log
I20260924 13:52:00.024816 4286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:52:00.012138 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36978 (local address 127.13.149.253:43403)
0924 13:52:00.012219 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0924 13:52:00.012222 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 13:52:00.012257 (+ 35us) server_negotiation.cc:407] Connection header received
0924 13:52:00.012323 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:52:00.012324 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:52:00.012336 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:52:00.012404 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:52:00.013113 (+ 709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:52:00.013867 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:52:00.014635 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:52:00.014902 (+ 267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:52:00.014954 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:52:00.019353 (+ 4399us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:52:00.019355 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:52:00.019356 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 13:52:00.019365 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:52:00.022263 (+ 2898us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:52:00.022713 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:52:00.022715 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:52:00.022716 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:52:00.022786 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:52:00.023019 (+ 233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:52:00.023020 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:52:00.023020 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:52:00.023169 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:52:00.023369 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0924 13:52:00.024620 (+ 1251us) server_negotiation.cc:299] Negotiation successful
0924 13:52:00.024710 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
Sep 24 13:52:00 dist-test-slave-sjl2 krb5kdc[4125](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257919, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.149.254@KRBTEST.COM for kudu/127.13.149.252@KRBTEST.COM
I20260924 13:52:00.028041 13911 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260924 13:52:00.028064 4356 raft_consensus.cc:348] T 00000000000000000000000000000000 P 41d8e6ee66994e6b90a42b5de18274ab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7e47f0da9d5438bb9f6c6a6914aa0d7" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 40347 } } peers { permanent_uuid: "4e0a602154dc454ab7defcc1ec52df99" member_type: VOTER last_known_addr { host: "127.13.149.253" port: 43403 } } peers { permanent_uuid: "41d8e6ee66994e6b90a42b5de18274ab" member_type: VOTER last_known_addr { host: "127.13.149.252" port: 35377 } }
I20260924 13:52:00.028250 4356 raft_consensus.cc:729] T 00000000000000000000000000000000 P 41d8e6ee66994e6b90a42b5de18274ab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41d8e6ee66994e6b90a42b5de18274ab, State: Initialized, Role: FOLLOWER
I20260924 13:52:00.028419 4356 consensus_queue.cc:260] T 00000000000000000000000000000000 P 41d8e6ee66994e6b90a42b5de18274ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7e47f0da9d5438bb9f6c6a6914aa0d7" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 40347 } } peers { permanent_uuid: "4e0a602154dc454ab7defcc1ec52df99" member_type: VOTER last_known_addr { host: "127.13.149.253" port: 43403 } } peers { permanent_uuid: "41d8e6ee66994e6b90a42b5de18274ab" member_type: VOTER last_known_addr { host: "127.13.149.252" port: 35377 } }
I20260924 13:52:00.028735 4363 sys_catalog.cc:455] T 00000000000000000000000000000000 P 41d8e6ee66994e6b90a42b5de18274ab [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7e47f0da9d5438bb9f6c6a6914aa0d7" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 40347 } } peers { permanent_uuid: "4e0a602154dc454ab7defcc1ec52df99" member_type: VOTER last_known_addr { host: "127.13.149.253" port: 43403 } } peers { permanent_uuid: "41d8e6ee66994e6b90a42b5de18274ab" member_type: VOTER last_known_addr { host: "127.13.149.252" port: 35377 } } }
I20260924 13:52:00.028867 4363 sys_catalog.cc:458] T 00000000000000000000000000000000 P 41d8e6ee66994e6b90a42b5de18274ab [sys.catalog]: This master's current role is: FOLLOWER
I20260924 13:52:00.028976 4356 sys_catalog.cc:564] T 00000000000000000000000000000000 P 41d8e6ee66994e6b90a42b5de18274ab [sys.catalog]: configured and running, proceeding with master startup.
I20260924 13:52:00.029026 4180 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "e7e47f0da9d5438bb9f6c6a6914aa0d7"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33906
I20260924 13:52:00.029915 4208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:52:00.019921 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.252:35377 (local address 127.0.0.1:34332)
0924 13:52:00.020041 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0924 13:52:00.020054 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0924 13:52:00.020096 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:52:00.020258 (+ 162us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:52:00.020259 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:52:00.020425 (+ 166us) client_negotiation.cc:189] Negotiated authn=SASL
0924 13:52:00.020482 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:52:00.020485 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:52:00.022618 (+ 2133us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:52:00.022621 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:52:00.023397 (+ 776us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:52:00.023402 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:52:00.023838 (+ 436us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:52:00.023839 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:52:00.023903 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:52:00.024490 (+ 587us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 13:52:00.024495 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 13:52:00.026041 (+ 1546us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 13:52:00.028112 (+ 2071us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 13:52:00.028113 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 13:52:00.028114 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 13:52:00.028349 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 13:52:00.028573 (+ 224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 13:52:00.028574 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 13:52:00.028574 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 13:52:00.028647 (+ 73us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 13:52:00.029231 (+ 584us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 13:52:00.029233 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 13:52:00.029477 (+ 244us) client_negotiation.cc:769] Sending connection context
0924 13:52:00.029590 (+ 113us) client_negotiation.cc:240] Negotiation successful
0924 13:52:00.029788 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":94}
I20260924 13:52:00.030464 4361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:52:00.019938 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34332 (local address 127.13.149.252:35377)
0924 13:52:00.020123 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0924 13:52:00.020125 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:52:00.020133 (+ 8us) server_negotiation.cc:407] Connection header received
0924 13:52:00.020154 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:52:00.020155 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:52:00.020166 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:52:00.020241 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:52:00.020560 (+ 319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:52:00.022500 (+ 1940us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:52:00.023464 (+ 964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:52:00.023759 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:52:00.023825 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:52:00.026312 (+ 2487us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:52:00.026314 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:52:00.026314 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 13:52:00.026323 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:52:00.028030 (+ 1707us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:52:00.028439 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:52:00.028440 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:52:00.028441 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:52:00.028509 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:52:00.028726 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:52:00.028727 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:52:00.028727 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:52:00.029125 (+ 398us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:52:00.030117 (+ 992us) server_negotiation.cc:1035] Waiting for connection context
0924 13:52:00.030188 (+ 71us) server_negotiation.cc:299] Negotiation successful
0924 13:52:00.030305 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":160,"thread_start_us":100,"threads_started":1}
W20260924 13:52:00.032677 4374 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 41d8e6ee66994e6b90a42b5de18274ab: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260924 13:52:00.032889 4374 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260924 13:52:00.033139 4206 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e7e47f0da9d5438bb9f6c6a6914aa0d7: Bootstrap starting.
I20260924 13:52:00.034185 4206 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e7e47f0da9d5438bb9f6c6a6914aa0d7: Neither blocks nor log segments found. Creating new log.
I20260924 13:52:00.034536 4206 log.cc:824] T 00000000000000000000000000000000 P e7e47f0da9d5438bb9f6c6a6914aa0d7: Log is configured to *not* fsync() on all Append() calls
I20260924 13:52:00.035321 4206 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e7e47f0da9d5438bb9f6c6a6914aa0d7: No bootstrap required, opened a new log
I20260924 13:52:00.032604 4288 master_runner.cc:186] Error getting master registration for 127.13.149.254:40347: OK, instance_id { permanent_uuid: "e7e47f0da9d5438bb9f6c6a6914aa0d7" instance_seqno: 1790257919677224 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260924 13:52:00.036043 4288 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260924 13:52:00.036882 4206 raft_consensus.cc:348] T 00000000000000000000000000000000 P e7e47f0da9d5438bb9f6c6a6914aa0d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7e47f0da9d5438bb9f6c6a6914aa0d7" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 40347 } } peers { permanent_uuid: "4e0a602154dc454ab7defcc1ec52df99" member_type: VOTER last_known_addr { host: "127.13.149.253" port: 43403 } } peers { permanent_uuid: "41d8e6ee66994e6b90a42b5de18274ab" member_type: VOTER last_known_addr { host: "127.13.149.252" port: 35377 } }
I20260924 13:52:00.037132 4206 raft_consensus.cc:729] T 00000000000000000000000000000000 P e7e47f0da9d5438bb9f6c6a6914aa0d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e7e47f0da9d5438bb9f6c6a6914aa0d7, State: Initialized, Role: FOLLOWER
I20260924 13:52:00.037235 4206 consensus_queue.cc:260] T 00000000000000000000000000000000 P e7e47f0da9d5438bb9f6c6a6914aa0d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7e47f0da9d5438bb9f6c6a6914aa0d7" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 40347 } } peers { permanent_uuid: "4e0a602154dc454ab7defcc1ec52df99" member_type: VOTER last_known_addr { host: "127.13.149.253" port: 43403 } } peers { permanent_uuid: "41d8e6ee66994e6b90a42b5de18274ab" member_type: VOTER last_known_addr { host: "127.13.149.252" port: 35377 } }
I20260924 13:52:00.037627 4206 sys_catalog.cc:564] T 00000000000000000000000000000000 P e7e47f0da9d5438bb9f6c6a6914aa0d7 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 13:52:00.037649 4180 raft_consensus.cc:482] T 00000000000000000000000000000000 P e7e47f0da9d5438bb9f6c6a6914aa0d7 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260924 13:52:00.037704 4180 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e7e47f0da9d5438bb9f6c6a6914aa0d7 [term 0 FOLLOWER]: Advancing to term 1
I20260924 13:52:00.038700 4180 raft_consensus.cc:504] T 00000000000000000000000000000000 P e7e47f0da9d5438bb9f6c6a6914aa0d7 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7e47f0da9d5438bb9f6c6a6914aa0d7" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 40347 } } peers { permanent_uuid: "4e0a602154dc454ab7defcc1ec52df99" member_type: VOTER last_known_addr { host: "127.13.149.253" port: 43403 } } peers { permanent_uuid: "41d8e6ee66994e6b90a42b5de18274ab" member_type: VOTER last_known_addr { host: "127.13.149.252" port: 35377 } }
I20260924 13:52:00.039011 4180 leader_election.cc:288] T 00000000000000000000000000000000 P e7e47f0da9d5438bb9f6c6a6914aa0d7 [CANDIDATE]: Term 1 election: Requested vote from peers 4e0a602154dc454ab7defcc1ec52df99 (127.13.149.253:43403), 41d8e6ee66994e6b90a42b5de18274ab (127.13.149.252:35377)
I20260924 13:52:00.039764 4330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e7e47f0da9d5438bb9f6c6a6914aa0d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "41d8e6ee66994e6b90a42b5de18274ab"
I20260924 13:52:00.039921 4330 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 41d8e6ee66994e6b90a42b5de18274ab [term 0 FOLLOWER]: Advancing to term 1
I20260924 13:52:00.041024 4330 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 41d8e6ee66994e6b90a42b5de18274ab [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7e47f0da9d5438bb9f6c6a6914aa0d7 in term 1.
I20260924 13:52:00.037588 4377 sys_catalog.cc:455] T 00000000000000000000000000000000 P e7e47f0da9d5438bb9f6c6a6914aa0d7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7e47f0da9d5438bb9f6c6a6914aa0d7" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 40347 } } peers { permanent_uuid: "4e0a602154dc454ab7defcc1ec52df99" member_type: VOTER last_known_addr { host: "127.13.149.253" port: 43403 } } peers { permanent_uuid: "41d8e6ee66994e6b90a42b5de18274ab" member_type: VOTER last_known_addr { host: "127.13.149.252" port: 35377 } } }
I20260924 13:52:00.042137 4377 sys_catalog.cc:458] T 00000000000000000000000000000000 P e7e47f0da9d5438bb9f6c6a6914aa0d7 [sys.catalog]: This master's current role is: FOLLOWER
I20260924 13:52:00.041396 4157 leader_election.cc:302] T 00000000000000000000000000000000 P e7e47f0da9d5438bb9f6c6a6914aa0d7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41d8e6ee66994e6b90a42b5de18274ab, e7e47f0da9d5438bb9f6c6a6914aa0d7; no voters:
I20260924 13:52:00.042415 4377 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e7e47f0da9d5438bb9f6c6a6914aa0d7 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 13:52:00.042586 4377 raft_consensus.cc:686] T 00000000000000000000000000000000 P e7e47f0da9d5438bb9f6c6a6914aa0d7 [term 1 LEADER]: Becoming Leader. State: Replica: e7e47f0da9d5438bb9f6c6a6914aa0d7, State: Running, Role: LEADER
I20260924 13:52:00.042692 4377 consensus_queue.cc:237] T 00000000000000000000000000000000 P e7e47f0da9d5438bb9f6c6a6914aa0d7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7e47f0da9d5438bb9f6c6a6914aa0d7" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 40347 } } peers { permanent_uuid: "4e0a602154dc454ab7defcc1ec52df99" member_type: VOTER last_known_addr { host: "127.13.149.253" port: 43403 } } peers { permanent_uuid: "41d8e6ee66994e6b90a42b5de18274ab" member_type: VOTER last_known_addr { host: "127.13.149.252" port: 35377 } }
I20260924 13:52:00.043200 4377 sys_catalog.cc:455] T 00000000000000000000000000000000 P e7e47f0da9d5438bb9f6c6a6914aa0d7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e7e47f0da9d5438bb9f6c6a6914aa0d7. Latest consensus state: current_term: 1 leader_uuid: "e7e47f0da9d5438bb9f6c6a6914aa0d7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7e47f0da9d5438bb9f6c6a6914aa0d7" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 40347 } } peers { permanent_uuid: "4e0a602154dc454ab7defcc1ec52df99" member_type: VOTER last_known_addr { host: "127.13.149.253" port: 43403 } } peers { permanent_uuid: "41d8e6ee66994e6b90a42b5de18274ab" member_type: VOTER last_known_addr { host: "127.13.149.252" port: 35377 } } }
I20260924 13:52:00.043280 4377 sys_catalog.cc:458] T 00000000000000000000000000000000 P e7e47f0da9d5438bb9f6c6a6914aa0d7 [sys.catalog]: This master's current role is: LEADER
W20260924 13:52:00.042742 4388 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e7e47f0da9d5438bb9f6c6a6914aa0d7: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260924 13:52:00.043955 4388 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260924 13:52:00.044013 4138 master_runner.cc:186] Error getting master registration for 127.13.149.253:43403: OK, instance_id { permanent_uuid: "4e0a602154dc454ab7defcc1ec52df99" instance_seqno: 1790257919835312 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260924 13:52:00.046542 4138 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260924 13:52:00.046693 4399 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 13:52:00.046929 4399 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 13:52:00.048838 4330 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 41d8e6ee66994e6b90a42b5de18274ab [term 1 FOLLOWER]: Refusing update from remote peer e7e47f0da9d5438bb9f6c6a6914aa0d7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 13:52:00.049507 4377 consensus_queue.cc:1035] T 00000000000000000000000000000000 P e7e47f0da9d5438bb9f6c6a6914aa0d7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "41d8e6ee66994e6b90a42b5de18274ab" member_type: VOTER last_known_addr { host: "127.13.149.252" port: 35377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 13:52:00.051795 4363 sys_catalog.cc:455] T 00000000000000000000000000000000 P 41d8e6ee66994e6b90a42b5de18274ab [sys.catalog]: SysCatalogTable state changed. Reason: New leader e7e47f0da9d5438bb9f6c6a6914aa0d7. Latest consensus state: current_term: 1 leader_uuid: "e7e47f0da9d5438bb9f6c6a6914aa0d7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7e47f0da9d5438bb9f6c6a6914aa0d7" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 40347 } } peers { permanent_uuid: "4e0a602154dc454ab7defcc1ec52df99" member_type: VOTER last_known_addr { host: "127.13.149.253" port: 43403 } } peers { permanent_uuid: "41d8e6ee66994e6b90a42b5de18274ab" member_type: VOTER last_known_addr { host: "127.13.149.252" port: 35377 } } }
I20260924 13:52:00.051993 4363 sys_catalog.cc:458] T 00000000000000000000000000000000 P 41d8e6ee66994e6b90a42b5de18274ab [sys.catalog]: This master's current role is: FOLLOWER
I20260924 13:52:00.053355 4209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:52:00.040591 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33920 (local address 127.13.149.254:40347)
0924 13:52:00.040638 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0924 13:52:00.040640 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:52:00.041145 (+ 505us) server_negotiation.cc:407] Connection header received
0924 13:52:00.041351 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:52:00.041352 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:52:00.041365 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:52:00.041397 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:52:00.042810 (+ 1413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:52:00.043633 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:52:00.044807 (+ 1174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:52:00.045127 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:52:00.045225 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:52:00.046894 (+ 1669us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:52:00.046895 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:52:00.046896 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 13:52:00.046903 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:52:00.048803 (+ 1900us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:52:00.049470 (+ 667us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:52:00.049471 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:52:00.049472 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:52:00.049554 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:52:00.052478 (+ 2924us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:52:00.052481 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:52:00.052482 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:52:00.052697 (+ 215us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:52:00.052801 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0924 13:52:00.053098 (+ 297us) server_negotiation.cc:299] Negotiation successful
0924 13:52:00.053233 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260924 13:52:00.054559 4377 sys_catalog.cc:455] T 00000000000000000000000000000000 P e7e47f0da9d5438bb9f6c6a6914aa0d7 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "e7e47f0da9d5438bb9f6c6a6914aa0d7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7e47f0da9d5438bb9f6c6a6914aa0d7" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 40347 } } peers { permanent_uuid: "4e0a602154dc454ab7defcc1ec52df99" member_type: VOTER last_known_addr { host: "127.13.149.253" port: 43403 } } peers { permanent_uuid: "41d8e6ee66994e6b90a42b5de18274ab" member_type: VOTER last_known_addr { host: "127.13.149.252" port: 35377 } } }
I20260924 13:52:00.054692 4377 sys_catalog.cc:458] T 00000000000000000000000000000000 P e7e47f0da9d5438bb9f6c6a6914aa0d7 [sys.catalog]: This master's current role is: LEADER
I20260924 13:52:00.055289 4363 sys_catalog.cc:455] T 00000000000000000000000000000000 P 41d8e6ee66994e6b90a42b5de18274ab [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "e7e47f0da9d5438bb9f6c6a6914aa0d7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7e47f0da9d5438bb9f6c6a6914aa0d7" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 40347 } } peers { permanent_uuid: "4e0a602154dc454ab7defcc1ec52df99" member_type: VOTER last_known_addr { host: "127.13.149.253" port: 43403 } } peers { permanent_uuid: "41d8e6ee66994e6b90a42b5de18274ab" member_type: VOTER last_known_addr { host: "127.13.149.252" port: 35377 } } }
I20260924 13:52:00.055424 4363 sys_catalog.cc:458] T 00000000000000000000000000000000 P 41d8e6ee66994e6b90a42b5de18274ab [sys.catalog]: This master's current role is: FOLLOWER
I20260924 13:52:00.055585 4400 mvcc.cc:205] Tried to move back new op lower bound from 7332896440516702208 to 7332896440495935488. Current Snapshot: MvccSnapshot[applied={T|T < 7332896440516702208}]
I20260924 13:52:00.056754 4286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:52:00.040770 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36980 (local address 127.13.149.253:43403)
0924 13:52:00.041214 (+ 444us) server_negotiation.cc:206] Beginning negotiation
0924 13:52:00.041217 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 13:52:00.047876 (+ 6659us) server_negotiation.cc:407] Connection header received
0924 13:52:00.047985 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:52:00.047987 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:52:00.048000 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:52:00.048031 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:52:00.048760 (+ 729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:52:00.049487 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:52:00.050720 (+ 1233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:52:00.051031 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:52:00.051114 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:52:00.052592 (+ 1478us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:52:00.052594 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:52:00.052595 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 13:52:00.052604 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:52:00.054272 (+ 1668us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:52:00.054998 (+ 726us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:52:00.055000 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:52:00.055000 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:52:00.055090 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:52:00.055868 (+ 778us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:52:00.055881 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:52:00.055881 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:52:00.056047 (+ 166us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:52:00.056203 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0924 13:52:00.056542 (+ 339us) server_negotiation.cc:299] Negotiation successful
0924 13:52:00.056649 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":413}
I20260924 13:52:00.056789 4401 mvcc.cc:205] Tried to move back new op lower bound from 7332896440516702208 to 7332896440495935488. Current Snapshot: MvccSnapshot[applied={T|T < 7332896440516702208}]
I20260924 13:52:00.065377 4361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:52:00.041151 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34336 (local address 127.13.149.252:35377)
0924 13:52:00.041259 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0924 13:52:00.041261 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:52:00.050846 (+ 9585us) server_negotiation.cc:407] Connection header received
0924 13:52:00.051470 (+ 624us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:52:00.051472 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:52:00.051483 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:52:00.051513 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:52:00.057828 (+ 6315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:52:00.058644 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:52:00.059709 (+ 1065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:52:00.060050 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:52:00.060134 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:52:00.061863 (+ 1729us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:52:00.061865 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:52:00.061865 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 13:52:00.061874 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:52:00.063831 (+ 1957us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:52:00.064275 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:52:00.064276 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:52:00.064276 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:52:00.064352 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:52:00.064569 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:52:00.064570 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:52:00.064570 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:52:00.064733 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:52:00.064849 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0924 13:52:00.065059 (+ 210us) server_negotiation.cc:299] Negotiation successful
0924 13:52:00.065153 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260924 13:52:00.065936 4399 catalog_manager.cc:1347] Generated new cluster ID: c48ebaf3f0274c3b84d22a1f6671a39f
I20260924 13:52:00.065999 4399 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 13:52:00.073746 4281 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.149.252" port: 35377 } attempt: 3
Sep 24 13:52:00 dist-test-slave-sjl2 krb5kdc[4125](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790257919, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.149.253@KRBTEST.COM for kudu/127.13.149.252@KRBTEST.COM
I20260924 13:52:00.091136 4284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:52:00.074134 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.149.252:35377 (local address 127.0.0.1:34350)
0924 13:52:00.074233 (+ 99us) negotiation.cc:107] Waiting for socket to connect
0924 13:52:00.074246 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0924 13:52:00.074288 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 13:52:00.074692 (+ 404us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 13:52:00.074693 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 13:52:00.074915 (+ 222us) client_negotiation.cc:189] Negotiated authn=SASL
0924 13:52:00.074974 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:52:00.074976 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:52:00.075962 (+ 986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:52:00.075963 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:52:00.076652 (+ 689us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 13:52:00.076655 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 13:52:00.077432 (+ 777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 13:52:00.077433 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 13:52:00.077507 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:52:00.078009 (+ 502us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 13:52:00.078015 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0924 13:52:00.079534 (+ 1519us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 13:52:00.081335 (+ 1801us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 13:52:00.081336 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 13:52:00.081337 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 13:52:00.081546 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 13:52:00.089249 (+ 7703us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 13:52:00.089251 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 13:52:00.089251 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 13:52:00.089341 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 13:52:00.090743 (+ 1402us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 13:52:00.090744 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 13:52:00.090809 (+ 65us) client_negotiation.cc:769] Sending connection context
0924 13:52:00.090873 (+ 64us) client_negotiation.cc:240] Negotiation successful
0924 13:52:00.091019 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":77}
I20260924 13:52:00.091645 4361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:52:00.074467 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34350 (local address 127.13.149.252:35377)
0924 13:52:00.074533 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0924 13:52:00.074535 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:52:00.074543 (+ 8us) server_negotiation.cc:407] Connection header received
0924 13:52:00.074586 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:52:00.074587 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:52:00.074597 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:52:00.075061 (+ 464us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:52:00.075113 (+ 52us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:52:00.075849 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:52:00.076803 (+ 954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:52:00.077295 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:52:00.078602 (+ 1307us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:52:00.079626 (+ 1024us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 13:52:00.079628 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 13:52:00.079628 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 13:52:00.079635 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 13:52:00.081263 (+ 1628us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:52:00.089003 (+ 7740us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:52:00.089006 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:52:00.089007 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:52:00.089140 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 13:52:00.089433 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 13:52:00.089434 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 13:52:00.089435 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 13:52:00.089652 (+ 217us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 13:52:00.089715 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0924 13:52:00.091449 (+ 1734us) server_negotiation.cc:299] Negotiation successful
0924 13:52:00.091539 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260924 13:52:00.093931 4281 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4e0a602154dc454ab7defcc1ec52df99: Bootstrap starting.
I20260924 13:52:00.094753 4281 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4e0a602154dc454ab7defcc1ec52df99: Neither blocks nor log segments found. Creating new log.
I20260924 13:52:00.095068 4281 log.cc:824] T 00000000000000000000000000000000 P 4e0a602154dc454ab7defcc1ec52df99: Log is configured to *not* fsync() on all Append() calls
I20260924 13:52:00.098109 4281 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4e0a602154dc454ab7defcc1ec52df99: No bootstrap required, opened a new log
I20260924 13:52:00.100315 4281 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4e0a602154dc454ab7defcc1ec52df99 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7e47f0da9d5438bb9f6c6a6914aa0d7" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 40347 } } peers { permanent_uuid: "4e0a602154dc454ab7defcc1ec52df99" member_type: VOTER last_known_addr { host: "127.13.149.253" port: 43403 } } peers { permanent_uuid: "41d8e6ee66994e6b90a42b5de18274ab" member_type: VOTER last_known_addr { host: "127.13.149.252" port: 35377 } }
I20260924 13:52:00.100492 4281 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4e0a602154dc454ab7defcc1ec52df99 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e0a602154dc454ab7defcc1ec52df99, State: Initialized, Role: FOLLOWER
I20260924 13:52:00.100574 4281 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4e0a602154dc454ab7defcc1ec52df99 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7e47f0da9d5438bb9f6c6a6914aa0d7" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 40347 } } peers { permanent_uuid: "4e0a602154dc454ab7defcc1ec52df99" member_type: VOTER last_known_addr { host: "127.13.149.253" port: 43403 } } peers { permanent_uuid: "41d8e6ee66994e6b90a42b5de18274ab" member_type: VOTER last_known_addr { host: "127.13.149.252" port: 35377 } }
I20260924 13:52:00.102236 4255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e7e47f0da9d5438bb9f6c6a6914aa0d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "4e0a602154dc454ab7defcc1ec52df99"
I20260924 13:52:00.102310 4254 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4e0a602154dc454ab7defcc1ec52df99 [term 0 FOLLOWER]: Advancing to term 1
I20260924 13:52:00.103564 4254 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 4e0a602154dc454ab7defcc1ec52df99 [term 1 FOLLOWER]: Refusing update from remote peer e7e47f0da9d5438bb9f6c6a6914aa0d7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 13:52:00.103935 4397 consensus_queue.cc:1035] T 00000000000000000000000000000000 P e7e47f0da9d5438bb9f6c6a6914aa0d7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0a602154dc454ab7defcc1ec52df99" member_type: VOTER last_known_addr { host: "127.13.149.253" port: 43403 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 13:52:00.104558 4255 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 4e0a602154dc454ab7defcc1ec52df99 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7e47f0da9d5438bb9f6c6a6914aa0d7 in term 1.
I20260924 13:52:00.102193 4281 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4e0a602154dc454ab7defcc1ec52df99 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 13:52:00.106778 4406 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4e0a602154dc454ab7defcc1ec52df99 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e7e47f0da9d5438bb9f6c6a6914aa0d7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7e47f0da9d5438bb9f6c6a6914aa0d7" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 40347 } } peers { permanent_uuid: "4e0a602154dc454ab7defcc1ec52df99" member_type: VOTER last_known_addr { host: "127.13.149.253" port: 43403 } } peers { permanent_uuid: "41d8e6ee66994e6b90a42b5de18274ab" member_type: VOTER last_known_addr { host: "127.13.149.252" port: 35377 } } }
I20260924 13:52:00.106956 4406 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4e0a602154dc454ab7defcc1ec52df99 [sys.catalog]: This master's current role is: FOLLOWER
I20260924 13:52:00.107062 4406 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4e0a602154dc454ab7defcc1ec52df99 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e7e47f0da9d5438bb9f6c6a6914aa0d7. Latest consensus state: current_term: 1 leader_uuid: "e7e47f0da9d5438bb9f6c6a6914aa0d7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7e47f0da9d5438bb9f6c6a6914aa0d7" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 40347 } } peers { permanent_uuid: "4e0a602154dc454ab7defcc1ec52df99" member_type: VOTER last_known_addr { host: "127.13.149.253" port: 43403 } } peers { permanent_uuid: "41d8e6ee66994e6b90a42b5de18274ab" member_type: VOTER last_known_addr { host: "127.13.149.252" port: 35377 } } }
I20260924 13:52:00.107177 4406 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4e0a602154dc454ab7defcc1ec52df99 [sys.catalog]: This master's current role is: FOLLOWER
I20260924 13:52:00.107249 4406 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4e0a602154dc454ab7defcc1ec52df99 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "e7e47f0da9d5438bb9f6c6a6914aa0d7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7e47f0da9d5438bb9f6c6a6914aa0d7" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 40347 } } peers { permanent_uuid: "4e0a602154dc454ab7defcc1ec52df99" member_type: VOTER last_known_addr { host: "127.13.149.253" port: 43403 } } peers { permanent_uuid: "41d8e6ee66994e6b90a42b5de18274ab" member_type: VOTER last_known_addr { host: "127.13.149.252" port: 35377 } } }
I20260924 13:52:00.107343 4406 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4e0a602154dc454ab7defcc1ec52df99 [sys.catalog]: This master's current role is: FOLLOWER
I20260924 13:52:00.109730 4397 sys_catalog.cc:455] T 00000000000000000000000000000000 P e7e47f0da9d5438bb9f6c6a6914aa0d7 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "e7e47f0da9d5438bb9f6c6a6914aa0d7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7e47f0da9d5438bb9f6c6a6914aa0d7" member_type: VOTER last_known_addr { host: "127.13.149.254" port: 40347 } } peers { permanent_uuid: "4e0a602154dc454ab7defcc1ec52df99" member_type: VOTER last_known_addr { host: "127.13.149.253" port: 43403 } } peers { permanent_uuid: "41d8e6ee66994e6b90a42b5de18274ab" member_type: VOTER last_known_addr { host: "127.13.149.252" port: 35377 } } }
I20260924 13:52:00.109843 4397 sys_catalog.cc:458] T 00000000000000000000000000000000 P e7e47f0da9d5438bb9f6c6a6914aa0d7 [sys.catalog]: This master's current role is: LEADER
I20260924 13:52:00.110525 4409 mvcc.cc:205] Tried to move back new op lower bound from 7332896440516702208 to 7332896440495935488. Current Snapshot: MvccSnapshot[applied={T|T < 7332896440516702208}]
I20260924 13:52:00.112960 4399 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 13:52:00.113622 4399 catalog_manager.cc:1504] Loading token signing keys...
I20260924 13:52:00.117952 4421 catalog_manager.cc:1259] Loaded cluster ID: c48ebaf3f0274c3b84d22a1f6671a39f
I20260924 13:52:00.118026 4421 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 4e0a602154dc454ab7defcc1ec52df99: loading cluster ID for follower catalog manager: success
I20260924 13:52:00.119177 4421 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 4e0a602154dc454ab7defcc1ec52df99: acquiring CA information for follower catalog manager: success
W20260924 13:52:00.119321 4421 catalog_manager.cc:873] Not found: no TSK found in the system table: failed to prepare follower catalog manager, will retry
I20260924 13:52:00.119524 4399 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e7e47f0da9d5438bb9f6c6a6914aa0d7: Generated new TSK 0
I20260924 13:52:00.119712 4399 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260924 13:52:00.143810 4286 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 13:52:00.142367 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36982 (local address 127.13.149.253:43403)
0924 13:52:00.142413 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0924 13:52:00.142415 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:52:00.143083 (+ 668us) server_negotiation.cc:407] Connection header received
0924 13:52:00.143201 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:52:00.143202 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:52:00.143214 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:52:00.143244 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:52:00.143731 (+ 487us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36982: BlockingRecv error: recv got EOF from 127.0.0.1:36982 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260924 13:52:00.144694 4361 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 13:52:00.142746 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34358 (local address 127.13.149.252:35377)
0924 13:52:00.142789 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0924 13:52:00.142791 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:52:00.144037 (+ 1246us) server_negotiation.cc:407] Connection header received
0924 13:52:00.144229 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:52:00.144230 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:52:00.144242 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:52:00.144271 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:52:00.144626 (+ 355us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34358: BlockingRecv error: recv got EOF from 127.0.0.1:34358 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260924 13:52:00.145695 4209 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 13:52:00.145057 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33930 (local address 127.13.149.254:40347)
0924 13:52:00.145102 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0924 13:52:00.145104 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:52:00.145145 (+ 41us) server_negotiation.cc:407] Connection header received
0924 13:52:00.145248 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:52:00.145249 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:52:00.145259 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:52:00.145287 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:52:00.145617 (+ 330us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33930: BlockingRecv error: recv got EOF from 127.0.0.1:33930 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
W20260924 13:52:00.150092 4286 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 13:52:00.149268 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36998 (local address 127.13.149.253:43403)
0924 13:52:00.149313 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0924 13:52:00.149316 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 13:52:00.149515 (+ 199us) server_negotiation.cc:407] Connection header received
0924 13:52:00.149630 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:52:00.149631 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:52:00.149653 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:52:00.149681 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:52:00.150034 (+ 353us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36998: BlockingRecv error: recv got EOF from 127.0.0.1:36998 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
W20260924 13:52:00.153389 4286 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 13:52:00.152470 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37006 (local address 127.13.149.253:43403)
0924 13:52:00.152510 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0924 13:52:00.152512 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:52:00.152775 (+ 263us) server_negotiation.cc:407] Connection header received
0924 13:52:00.152870 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:52:00.152871 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:52:00.152882 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:52:00.152910 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:52:00.153330 (+ 420us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37006: BlockingRecv error: recv got EOF from 127.0.0.1:37006 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260924 13:52:00.157428 4286 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 13:52:00.156528 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37016 (local address 127.13.149.253:43403)
0924 13:52:00.156568 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0924 13:52:00.156570 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:52:00.156825 (+ 255us) server_negotiation.cc:407] Connection header received
0924 13:52:00.156958 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:52:00.156959 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:52:00.156971 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:52:00.157001 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:52:00.157368 (+ 367us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37016: BlockingRecv error: recv got EOF from 127.0.0.1:37016 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260924 13:52:00.163527 4286 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 13:52:00.162608 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37018 (local address 127.13.149.253:43403)
0924 13:52:00.162650 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0924 13:52:00.162652 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:52:00.162929 (+ 277us) server_negotiation.cc:407] Connection header received
0924 13:52:00.163027 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:52:00.163029 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:52:00.163040 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:52:00.163070 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:52:00.163453 (+ 383us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37018: BlockingRecv error: recv got EOF from 127.0.0.1:37018 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260924 13:52:00.174182 4286 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 13:52:00.173080 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37032 (local address 127.13.149.253:43403)
0924 13:52:00.173131 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0924 13:52:00.173133 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:52:00.173549 (+ 416us) server_negotiation.cc:407] Connection header received
0924 13:52:00.173654 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:52:00.173655 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:52:00.173668 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:52:00.173699 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:52:00.174104 (+ 405us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37032: BlockingRecv error: recv got EOF from 127.0.0.1:37032 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
W20260924 13:52:00.192415 4286 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 13:52:00.191484 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37044 (local address 127.13.149.253:43403)
0924 13:52:00.191533 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0924 13:52:00.191535 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:52:00.191828 (+ 293us) server_negotiation.cc:407] Connection header received
0924 13:52:00.191927 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:52:00.191928 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:52:00.191940 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:52:00.191968 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:52:00.192355 (+ 387us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37044: BlockingRecv error: recv got EOF from 127.0.0.1:37044 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260924 13:52:00.226912 4286 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 13:52:00.225851 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37052 (local address 127.13.149.253:43403)
0924 13:52:00.225903 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0924 13:52:00.225906 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 13:52:00.226236 (+ 330us) server_negotiation.cc:407] Connection header received
0924 13:52:00.226353 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:52:00.226354 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:52:00.226367 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:52:00.226400 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:52:00.226847 (+ 447us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37052: BlockingRecv error: recv got EOF from 127.0.0.1:37052 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
W20260924 13:52:00.293238 4286 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 13:52:00.292460 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37066 (local address 127.13.149.253:43403)
0924 13:52:00.292541 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0924 13:52:00.292543 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:52:00.292552 (+ 9us) server_negotiation.cc:407] Connection header received
0924 13:52:00.292580 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:52:00.292581 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:52:00.292590 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:52:00.292687 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:52:00.293174 (+ 487us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37066: BlockingRecv error: recv got EOF from 127.0.0.1:37066 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
W20260924 13:52:00.423130 4286 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 13:52:00.422455 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37072 (local address 127.13.149.253:43403)
0924 13:52:00.422517 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0924 13:52:00.422519 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:52:00.422597 (+ 78us) server_negotiation.cc:407] Connection header received
0924 13:52:00.422672 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:52:00.422673 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:52:00.422685 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:52:00.422748 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:52:00.423072 (+ 324us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37072: BlockingRecv error: recv got EOF from 127.0.0.1:37072 (error 108)
Metrics: {"server-negotiator.queue_time_us":34}
W20260924 13:52:00.681455 4286 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 13:52:00.680644 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37082 (local address 127.13.149.253:43403)
0924 13:52:00.680704 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0924 13:52:00.680711 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0924 13:52:00.680755 (+ 44us) server_negotiation.cc:407] Connection header received
0924 13:52:00.680843 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:52:00.680844 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:52:00.680858 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:52:00.680906 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0924 13:52:00.681398 (+ 492us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37082: BlockingRecv error: recv got EOF from 127.0.0.1:37082 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
I20260924 13:52:01.033499 4374 catalog_manager.cc:1259] Loaded cluster ID: c48ebaf3f0274c3b84d22a1f6671a39f
I20260924 13:52:01.033581 4374 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 41d8e6ee66994e6b90a42b5de18274ab: loading cluster ID for follower catalog manager: success
I20260924 13:52:01.034381 4374 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 41d8e6ee66994e6b90a42b5de18274ab: acquiring CA information for follower catalog manager: success
I20260924 13:52:01.034680 4374 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 41d8e6ee66994e6b90a42b5de18274ab: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260924 13:52:01.119963 4421 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 4e0a602154dc454ab7defcc1ec52df99: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260924 13:52:01.197252 4489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:52:01.194581 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37086 (local address 127.13.149.253:43403)
0924 13:52:01.194694 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0924 13:52:01.194695 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0924 13:52:01.194706 (+ 11us) server_negotiation.cc:407] Connection header received
0924 13:52:01.194733 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:52:01.194733 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:52:01.194781 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:52:01.194865 (+ 84us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 13:52:01.194984 (+ 119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:52:01.195544 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:52:01.196390 (+ 846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:52:01.196598 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:52:01.196659 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:52:01.196894 (+ 235us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 13:52:01.197040 (+ 146us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 13:52:01.197118 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0924 13:52:01.197149 (+ 31us) server_negotiation.cc:299] Negotiation successful
0924 13:52:01.197170 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92,"thread_start_us":55,"threads_started":1}
I20260924 13:52:01.201673 4495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:52:01.199227 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34370 (local address 127.13.149.252:35377)
0924 13:52:01.199369 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0924 13:52:01.199370 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0924 13:52:01.199379 (+ 9us) server_negotiation.cc:407] Connection header received
0924 13:52:01.199420 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:52:01.199421 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:52:01.199459 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:52:01.199526 (+ 67us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 13:52:01.199624 (+ 98us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:52:01.200177 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:52:01.200958 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:52:01.201152 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:52:01.201205 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:52:01.201349 (+ 144us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 13:52:01.201460 (+ 111us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 13:52:01.201524 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0924 13:52:01.201562 (+ 38us) server_negotiation.cc:299] Negotiation successful
0924 13:52:01.201584 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":118,"thread_start_us":74,"threads_started":1}
I20260924 13:52:01.205892 4495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:52:01.203413 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34386 (local address 127.13.149.252:35377)
0924 13:52:01.203445 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0924 13:52:01.203447 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:52:01.203474 (+ 27us) server_negotiation.cc:407] Connection header received
0924 13:52:01.203526 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:52:01.203526 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:52:01.203535 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:52:01.203579 (+ 44us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 13:52:01.203729 (+ 150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:52:01.204180 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:52:01.205149 (+ 969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:52:01.205405 (+ 256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:52:01.205474 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:52:01.205606 (+ 132us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 13:52:01.205641 (+ 35us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 13:52:01.205682 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0924 13:52:01.205759 (+ 77us) server_negotiation.cc:299] Negotiation successful
0924 13:52:01.205791 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260924 13:52:01.206113 4489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:52:01.203201 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37102 (local address 127.13.149.253:43403)
0924 13:52:01.203283 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0924 13:52:01.203284 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0924 13:52:01.203393 (+ 109us) server_negotiation.cc:407] Connection header received
0924 13:52:01.203492 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:52:01.203493 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:52:01.203504 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:52:01.203531 (+ 27us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 13:52:01.203734 (+ 203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:52:01.204451 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:52:01.205380 (+ 929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:52:01.205670 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:52:01.205736 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:52:01.205851 (+ 115us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 13:52:01.205883 (+ 32us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 13:52:01.205922 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0924 13:52:01.206015 (+ 93us) server_negotiation.cc:299] Negotiation successful
0924 13:52:01.206041 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260924 13:52:01.207192 4502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 13:52:01.203168 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33932 (local address 127.13.149.254:40347)
0924 13:52:01.203265 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0924 13:52:01.203267 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 13:52:01.203398 (+ 131us) server_negotiation.cc:407] Connection header received
0924 13:52:01.203451 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 13:52:01.203452 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 13:52:01.203506 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 13:52:01.203582 (+ 76us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 13:52:01.203681 (+ 99us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:52:01.204461 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:52:01.206431 (+ 1970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 13:52:01.206674 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 13:52:01.206728 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 13:52:01.206897 (+ 169us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 13:52:01.206987 (+ 90us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 13:52:01.207037 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0924 13:52:01.207091 (+ 54us) server_negotiation.cc:299] Negotiation successful
0924 13:52:01.207115 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79,"thread_start_us":46,"threads_started":1}
I20260924 13:52:01.208758 13911 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJN9WRF/build/release/bin/kudu with pid 4138
I20260924 13:52:01.213713 13911 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJN9WRF/build/release/bin/kudu with pid 4213
I20260924 13:52:01.218873 13911 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJN9WRF/build/release/bin/kudu with pid 4288
2026-09-24T13:52:01Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (2743 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (5462 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (292184 ms total)
[ PASSED ] 54 tests.
[ FAILED ] 2 tests, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
[ FAILED ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
2 FAILED TESTS